Storing Cookies (See : http://ec.europa.eu/ipg/basics/legal/cookies/index_en.htm ) help us to bring you our services at overunity.com . If you use this website and our services you declare yourself okay with using cookies .More Infos here:
https://overunity.com/5553/privacy-policy/
If you do not agree with storing cookies, please LEAVE this website now. From the 25th of May 2018, every existing user has to accept the GDPR agreement at first login. If a user is unwilling to accept the GDPR, he should email us and request to erase his account. Many thanks for your understanding

User Menu

Custom Search

Author Topic: Let's crack Sloot algorithm - infinite "compression"  (Read 24934 times)

nix85

  • Hero Member
  • *****
  • Posts: 1431
Let's crack Sloot algorithm - infinite "compression"
« on: July 16, 2020, 06:57:03 PM »
i know this is not overunity, but it kinda is..

it has already been discussed here https://overunity.com/7456/another-strange-story/

in case you didn't hear about it, jan slooth was TV tehnician from netherlands who after 20 years of research in 1995 created a data sharing technique which could allegedly store a complete movie in 8 kilobytes. 1 day before he would hand over the code an would be a multimillionaire he died of a "heartattack"

he allegedly played 16 movies simultaneously from a 64kb phone card

he said it was "so simple a simple tehnician could figure it in a second if shown the source code"

he was EXTREMELY paranoid and secretive about it

>> https://www.youtube.com/watch?v=jLp5gS9h9UI

data to be sent was 2 million times smaller than the original.

if taken literally, this would be like to squeeze lets say 1999999 various numbers into 1 digit. impossible like that of course, information entropy..

...but is it?

 i'll first quote a guy that left a comment here http://jansloot.telcomsoft.nl/Sources-1/More/CaptainCosmos/Not_Compression.htm#.Xdyn0PlKiCq

Quote
You're allmost there. Sloot just used an analog converter. The analog signal was created by a device that was parametrized by the data on the chipcard. The precise digital measuring of the output of the analog signal then generates the digital sequence. Digital compressors are extremely limited because computers use integer or real data to work with. Analog systems can contain millions (some unlimited) of times the quantity of information that digital systems contain.

Sloot was a tv-technician in the transition period from high-quality analog technology to the digitalisation. He made the link between the two. His technique would or will ruin a whole industry, making ''our'' hich-tech IT based on digital processing completely obsolete.

For the one who does not understand the upper part: Consider a circle. The relation between diameter and circumference is the nubmer PI. Thats, 3.1415... and an inifinite numer of digits never recurring. Thats quite an amount of information in a simple analog thing.

There are other simple analog things that contain lots of information, even if you stay in the ''rational number'' range. Every movie, ever made on DVD and goïng to be made on DVD or every file on a computer in the world can be drawn by a single line on one sheet of paper: the slope (Y/X) as a finite row of digits can be the result of a digital measurement. Replace the sheet with a (virtual) tv-screen, and you have the SLOOT-compression, or something like. It's a digital to analog- and reverse thing. No more no less.

Sloot himself spoke about the ''end of the digital era''.

but does it really have to be analog?

sloot was generating these tiny keys by comparing source file to a database, 70mb for each type of data, videos, sound, text, pictures, and reverse was done one the other end.

Roel Pieper, former CTO and board member of Philips, is quoted as saying (translated from Dutch):

Quote
It is not about compression. Everyone is mistaken about that. The principle can be compared with a concept as Adobe-postscript, where sender and receiver know what kind of data recipes can be transferred, without the data itself actually being sent.

indeed, there is no compression here.

remember that since data is so extremelly "reduced", same 8kb key must be able to represent many different sets of data, different movies. but how.

WITH DYNAMIC COMPONENTS ON BOTH ENCODING AND DECODING SIDE, PARAMETRIZED BY SIMPLE SET OF VARIABLES ON THE BEGINNING OF THE KEY.

this is the only way this could have worked, even theoretically

to put it most plainly, we can represent any file of any kind as simply a sequence of numbers, so goal is to reduce 1 page full of random numbers to let's say 3/4 or 2/3 or even 1/2 of the page without data lost, with multiple iterations near infinite "compression" results. but how.

like i said, if we have a certain mechanism generating random data, let's say 10 generators of random numbers on encoder and decoder side.

lets say when movie is about to be turned into a 8kb key, those random generators are parametrized by exact time and date number, so its always unique.

totally unique and random datastream will be produced by the oscillators, THE SAME datastream on both sides.

we also have the same algorithm and same 70mb (can be more) library on both sides.

70mb library was probably an index of random numbers and algorithm had to first recognize which chunks of generated random numbers correspond to the source file..

maybe then it just sent the number of the table in which the big number was to be found and in combination with the timestamp software on the other side could look into that table, compare it with its own datastream at that timestamp moment and extract the usable data.

this is just a crude idea, probably not the best one, but i believe it was somewhere along these lines.

i also stumbled upon this old video, is it fake i dont know, claiming to reduce any file to 256 bytes

https://www.youtube.com/watch?v=kQsWP6n03EU

as sloot himself said it was extremely simple

nix85

  • Hero Member
  • *****
  • Posts: 1431
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #1 on: July 17, 2020, 01:52:57 AM »
...

WhatIsIt

  • Hero Member
  • *****
  • Posts: 651
    • At The End It Will Matter!
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #2 on: July 17, 2020, 03:15:02 AM »
Probably will not be right, but can try.

Letters are 32 bit numbers each, 1 integer.


We have sequence ABCD, which are 4 * 32 bit number (integers),

but it has only 256 iterations, which we can write in 1 integer or less

For combination AAAA, we write number 0, and max is 255 (DDDD).


Can we apply this further?



Because there are 26 letters, number of iterations (combinations) for ABCD will be 456976 for all possible letters in that 4 symbol block,

26 * 26 * 26 * 26 = 456976

and 456976 we can also write inside 1 integer instead of 4 integers.

AAAA = 0

ZZZZ = 456975


So, we write number of sequence (iterations) of blocks of 4 or more symbols, rather than integer symbols.

And if we have in small database all iterations, we can simply reverse number to given block of 4 letters.



I said I will try only, not claiming it will work.

It will work only for limited number of symbols.


Same principle goes for number of colors of 1 pixel.

Digital to analog, or maybe vice versa?

nix85

  • Hero Member
  • *****
  • Posts: 1431
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #3 on: July 17, 2020, 05:49:34 AM »
that was one of first things i thought of, how many bits per letter depends on the encoding system

Quote
An ASCII character in 8-bit ASCII encoding is 8 bits (1 byte), though it can fit in 7 bits.

An ISO-8895-1 character in ISO-8859-1 encoding is 8 bits (1 byte).

A Unicode character in UTF-8 encoding is between 8 bits (1 byte) and 32 bits (4 bytes).

A Unicode character in UTF-16 encoding is between 16 (2 bytes) and 32 bits (4 bytes), though most of the common characters take 16 bits. This is the encoding used by Windows internally.

A Unicode character in UTF-32 encoding is always 32 bits (4 bytes).

An ASCII character in UTF-8 is 8 bits (1 byte), and in UTF-16 - 16 bits.

The additional (non-ASCII) characters in ISO-8895-1 (0xA0-0xFF) would take 16 bits in UTF-8 and UTF-16.

i gave a lot of thought to extended ascii II table, each of the 8 bit characters may represent a number from 0-255, or 255-510, or 510-765, or 765-1020, or a certain function or coordinate of a cube encoding large number of variables.

https://theasciicode.com.ar/

for example 255x255x255 cube contains 16,581,375 cells and any of those cells can be addressed with just 24 bits (or even 21).

of course there is no reason to limit this to 3d cube, we could give it 30 dimensions instead so string of 30 ascii characters would be enough to address any of this many variables

1,571,105,731,713,312,715,511,913,444,948,824,285,516,982,702,388,429,082,930,088,043,212,890,625

but that approach has great limitation that it does not allow for further compression, unlike if we got a string of numbers that could be further reduced.

this is one of brute force approaches, not a solution by itself. solution might be combination of this with some complex logic and random data stream. receiver knows "what can and cannot be sent".


WhatIsIt

  • Hero Member
  • *****
  • Posts: 651
    • At The End It Will Matter!
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #4 on: July 17, 2020, 06:21:20 AM »
Yes,

You saw it well, it can not be compressed further.

nix85

  • Hero Member
  • *****
  • Posts: 1431
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #5 on: July 17, 2020, 07:15:37 AM »
if we meditated on this for some time, im sure we would crack it, but i guess no one really cares that much. i think such revelations come by merit, soot got it after 20 years of search, how impatient would it be to expect to get it in few days or weeks invested in this. but implications are too big to be forgotten, hundreds of ultraHD movies on a floppy disk, maybe world isnt ready for it.

WhatIsIt

  • Hero Member
  • *****
  • Posts: 651
    • At The End It Will Matter!
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #6 on: July 17, 2020, 07:49:16 AM »
You got me thinking, and the aplications of such invention are huge.
You can have Wikipedia on your phone without net.
Movies, books, everything.

Tons of data, almost unlimited.

ramset

  • Hero Member
  • *****
  • Posts: 8073
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #7 on: July 17, 2020, 01:30:29 PM »
Being autonomous of the net and having worlds knowledge available in such small package [pure unaltered by ??]  would truly be a gift towards independence and for humanity .

nix85

  • Hero Member
  • *****
  • Posts: 1431
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #8 on: July 17, 2020, 05:57:39 PM »
something i thought of yesterday, not a solution, just a thought.

something like criss cross

Suppose for example that you have a table of 1 digit numbers 10 x 10. Once the random generator fills the table with first set of 100 numbers, you can now create 40 unique 10 digit sequences from these 100 just by reading first line from left to right then right to left, same for other 9 lines, then same vertically for all 10 columns, then you can add at least 30 diagonal lines containing ~10 digits on average

so we have aproximatelly 700 digit unique sequence and any sequence longer than 4 digits that cooresponds to source file within those 10 digit chunks can be addressed with just 4 digit coordinates, start and stop cell.

for example coordinates 5055 can address a 5 or 6 digit sequence

with multiple iterations sequence of coordinates would be generated that should be shorter than source string.

but it probably would not be, what are the chances 5 or more digit sequences from the source string would appear among 70 ~10 digit random numbers strings

also, problem with this is you must place a non-numerical separator so algorithm knows when set of random numbers changes, so output would not be pure shorter numbers, but would have these separators among them for each iteration but maybe it could memorize their location and remove them so pure number string can be further reduced

just an idea. not such a good idea

he used a 70Mb library for each type of data. We do not know if that library contained just index of random numbers, or complex logic or both.


nix85

  • Hero Member
  • *****
  • Posts: 1431
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #9 on: July 17, 2020, 07:31:05 PM »
...

WhatIsIt

  • Hero Member
  • *****
  • Posts: 651
    • At The End It Will Matter!
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #10 on: July 17, 2020, 11:07:01 PM »
something i thought of yesterday, not a solution, just a thought.

something like criss cross

Suppose for example that you have a table of 1 digit numbers 10 x 10. Once the random generator fills the table with first set of 100 numbers, you can now create 40 unique 10 digit sequences from these 100 just by reading first line from left to right then right to left, same for other 9 lines, then same vertically for all 10 columns, then you can add at least 30 diagonal lines containing ~10 digits on average

so we have aproximatelly 700 digit unique sequence and any sequence longer than 4 digits that cooresponds to source file within those 10 digit chunks can be addressed with just 4 digit coordinates, start and stop cell.

for example coordinates 5055 can address a 5 or 6 digit sequence

with multiple iterations sequence of coordinates would be generated that should be shorter than source string.

but it probably would not be, what are the chances 5 or more digit sequences from the source string would appear among 70 ~10 digit random numbers strings

also, problem with this is you must place a non-numerical separator so algorithm knows when set of random numbers changes, so output would not be pure shorter numbers, but would have these separators among them for each iteration but maybe it could memorize their location and remove them so pure number string can be further reduced

just an idea. not such a good idea

he used a 70Mb library for each type of data. We do not know if that library contained just index of random numbers, or complex logic or both.


It is good idea actually.

If the table is big enough to cover most of possible number combinations, could work.

This way you can write even big numbers with just coordinates.


You are not far away from his idea.


In his 70Mb he probably had some kind of similar table for creating strings or sequences of numbers,

which are larger than pointers, vectors itself (coordinates in your case).



WhatIsIt

  • Hero Member
  • *****
  • Posts: 651
    • At The End It Will Matter!
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #11 on: July 17, 2020, 11:13:30 PM »
something i thought of yesterday, not a solution, just a thought.

something like criss cross

Suppose for example that you have a table of 1 digit numbers 10 x 10. Once the random generator fills the table with first set of 100 numbers, you can now create 40 unique 10 digit sequences from these 100 just by reading first line from left to right then right to left, same for other 9 lines, then same vertically for all 10 columns, then you can add at least 30 diagonal lines containing ~10 digits on average

so we have aproximatelly 700 digit unique sequence and any sequence longer than 4 digits that cooresponds to source file within those 10 digit chunks can be addressed with just 4 digit coordinates, start and stop cell.

for example coordinates 5055 can address a 5 or 6 digit sequence

with multiple iterations sequence of coordinates would be generated that should be shorter than source string.

but it probably would not be, what are the chances 5 or more digit sequences from the source string would appear among 70 ~10 digit random numbers strings

also, problem with this is you must place a non-numerical separator so algorithm knows when set of random numbers changes, so output would not be pure shorter numbers, but would have these separators among them for each iteration but maybe it could memorize their location and remove them so pure number string can be further reduced

just an idea. not such a good idea

he used a 70Mb library for each type of data. We do not know if that library contained just index of random numbers, or complex logic or both.


And one more thing.


I know you showed example.

But in example are decimal numbers, so we need 10 symbols to cover.

What about same table with just binary 0 or 1. Lots of sequences.

Will that be simpler or more complicated?

nix85

  • Hero Member
  • *****
  • Posts: 1431
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #12 on: July 17, 2020, 11:50:09 PM »

It is good idea actually.

If the table is big enough to cover most of possible number combinations, could work.

This way you can write even big numbers with just coordinates.


You are not far away from his idea.


In his 70Mb he probably had some kind of similar table for creating strings or sequences of numbers,

which are larger than pointers, vectors itself (coordinates in your case).

i was just trying to be humble :) its a good idea, but i dont think it would work like that.

i was thinking also about big table and other data than single digit numbers, including ascii, just alphabet and the binary you mention. with binary i think it would be less efficient.

for example whole source file could be one gigantic table, and at each set of random numbers, algorithm would find all the sequences in the source table that match those in the random table and replace them with coordinates, but then u must place a spacer before and after coordinates and what is the gain then, u now got spacers in the output which cannot be removed unless u also memorized coordinates for them which cost a lot.

there is more to this, i think he used complex logic so reciever knows "what can and cannot be sent"...

or maybe library was really just index of random numbers and he found some clever way to address them apparently breaking the information entropy. but he clearly did not break it, he found a workaround.




WhatIsIt

  • Hero Member
  • *****
  • Posts: 651
    • At The End It Will Matter!
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #13 on: July 17, 2020, 11:59:09 PM »

also, problem with this is you must place a non-numerical separator so algorithm knows when set of random numbers changes, so output would not be pure shorter numbers, but would have these separators among them for each iteration but maybe it could memorize their location and remove them so pure number string can be further reduced



If you don't stop, separator, delimiter is not needed.

First coordinate, begin

Second coordinate, end sequence (2345)

Third coordinate, begin second sequence,

Fourth coordinate, end second sequence (12345),

But, actually you read it from memory as 234512345, as one sequence without delimiter,

just search in table for pieces which are longest to fill sequence.


You read it as continuous number, just as it is in memory.



You read memory from beginning till  the end as series of numbers merged together.

We know how long byte or integer is.


For example we have bytes

104

and

055

merge it, we know that byte is 3 places,

104055 and then search for best suited sequence to fill it from table,

I merged 2 bytes, but you can merge 100 bytes as continuous sequence and then look into table.


The whole movie is one big continuous number, sequence.

When you look memory with hex editor, there is number after number. You can look at it as one long number.


Then you don't need delimiter.


nix85

  • Hero Member
  • *****
  • Posts: 1431
Re: Let's crack Sloot algorithm - infinite "compression"
« Reply #14 on: July 18, 2020, 12:27:59 AM »

If you don't stop, separator, delimiter is not needed.

First coordinate, begin

Second coordinate, end sequence (2345)

Third coordinate, begin second sequence,

Fourth coordinate, end second sequence (12345),

But, actually you read it from memory as 234512345, as one sequence without delimiter,

just search in table for pieces which are longest to fill sequence.


You read it as continuous number, just as it is in memory.



You read memory from beginning till  the end as series of numbers merged together.

We know how long byte or integer is.


For example we have bytes

104

and

255

merge it, we know that byte is 3 places,

104255 and then search for best suited sequence to fill it from table,

I merged 2 bytes, but you can merge 100 bytes as continuous sequence and then look into table.


Then you don't need delimiter.

if you encode the source linearly, from beginning onward, you need a spacer, not all iterations of the 100 random numbers will yield same number of matches, some iterations will yield 10 pairs of coordinates, some none, and you must know when random numbers change.

in a case if you go about it non-linearly, turning the whole source file into a table, and for each set of 100 random numbers finding the matches across the whole source file and replacing them with coordinates ..you again need spacers (or something to replace them).


lets say after 10,000 iterations you have turned a whole source file into coordinates...

encoding algorithm could of course know which sequences have been replaced with coordinates at which iteration, but how would algorithm on decoding side know.

no way for it to know, it must know exactly at which iteration those coordinates where placed there.