Author Topic: String Compression and Decompression  (Read 937 times)

Offline John

  • Forum Support / SB Dev
  • Posts: 2879
    • ScriptBasic Open Source Project
String Compression and Decompression
« on: December 17, 2018, 05:37:50 PM »
Peter on the BaCon forum is experimenting with string compression.

compression and decompression

Here is the Script BASIC solution.

Code: Script BASIC
  1. IMPORT t.bas
  2. IMPORT zlib.bas
  3.  
  4. PRINT FORMAT("War & Peace uncompressed is %~#,###,###~ bytes. Compressed it's %~#,###,###~ bytes.\n", _
  5.       FILELEN("warpeace.txt"), LEN(zlib::Compress(t::LoadString("warpeace.txt"))))
  6.  


$ time scriba wpc.sb
War & Peace uncompressed is 3,202,941 bytes. Compressed it's 1,195,214 bytes.

real   0m0.385s
user   0m0.377s
sys   0m0.008s
$

« Last Edit: December 17, 2018, 08:38:31 PM by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 2879
    • ScriptBasic Open Source Project
Re: String Compression and Decompression
« Reply #1 on: December 17, 2018, 09:12:06 PM »
Here is a file compress and uncompress example.

The file being compressed is deleted with the gzip() function, You may want to make a backup before playing with this.

You can use standard Linux utilities to uncompress files compressed with the zlib extension module.

Code: Script BASIC
  1. IMPORT zlib.bas
  2.  
  3. PRINT FILELEN("warpeace.txt"),"\n"
  4. zlib::gzip "warpeace.txt", "warpeace.gz"
  5. PRINT FILELEN("warpeace.gz"),"\n"
  6. zlib::gunzip("warpeace.gz", "warpeace.txt")
  7. PRINT FILELEN("warpeace.txt"),"\n"
  8.  


$ time scriba unwpc.sb
3202941
1195226
3202941

real   0m0.450s
user   0m0.433s
sys   0m0.016s
$

« Last Edit: December 19, 2018, 01:58:55 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 741
Re: String Compression and Decompression
« Reply #2 on: December 18, 2018, 10:00:39 PM »
Using MBC and libzip(uses zlib):

Code: Text
  1. $MODULE ZIP.mbc
  2. $EXECON
  3.  
  4.  
  5. dim fname$,ret
  6.  
  7. if G_argc < 2 then
  8.     print "usage: ",command$(0)," archive"
  9.     end = 1
  10. end if
  11.  
  12. fname$ = command$(1)
  13.  
  14. print E"Source file size:\t";lof%(fname$)/1000;"kB"
  15.  
  16. ret = Compress(fname$,"W&P.zip")
  17. if ret then
  18.     print E"Compressed file size:\t",ret/1000,"kB"
  19. else
  20.     end = 1
  21. end if
  22.  
  23. ret = Extract("W&P.zip","UNPACKED-W&P.txt")
  24. if ret then
  25.     print E"Extracted  file size:\t",ret/1000,"kB"
  26. end if

Code: [Select]
[riveraa@MacDev ~/Projects/mbc/compression] $ time ./ziptest "war and peace.txt"
Source file size: 3359kB
Compressed file size: 1220kB
Extracted  file size: 3359kB

real 0m0.417s
user 0m0.404s
sys 0m0.009s

AIR.

Offline John

  • Forum Support / SB Dev
  • Posts: 2879
    • ScriptBasic Open Source Project
Re: String Compression and Decompression
« Reply #3 on: December 18, 2018, 11:32:15 PM »
MBC sure looks easier to use than BaCon with its C helper dependencies.

Offline AIR

  • BASIC Developer
  • Posts: 741
Re: String Compression and Decompression
« Reply #4 on: March 07, 2019, 02:15:19 AM »
USING GO

Code: Go
  1. package main
  2.  
  3. import (
  4.         "fmt"
  5.         "log"
  6.         "os"
  7.  
  8.         "github.com/dustin/go-humanize"
  9.         "github.com/mholt/archiver"
  10. )
  11.  
  12. func main() {
  13.         // List of Files to Zip
  14.         files := []string{"warpeace.txt"}
  15.         output := "warpeace.zip"
  16.         output2 := "warpeace.txt"
  17.  
  18.         if err := archiver.Archive(files, output); err != nil {
  19.                 log.Fatal(err)
  20.         }
  21.  
  22.         if err := archiver.Unarchive(output, "test"); err != nil {
  23.                 log.Fatal(err)
  24.         }
  25.         fmt.Println("Source File Size:", humanize.Bytes(getFileSize("warpeace.txt")))
  26.         fmt.Println("Compressed File Size:", humanize.Bytes(getFileSize(output)))
  27.         fmt.Println("Extracted File Size:", humanize.Bytes(getFileSize("test/"+output2)))
  28.  
  29. }
  30.  
  31. func getFileSize(filename string) uint64 {
  32.         file, err := os.Open(filename)
  33.         if err != nil {
  34.                 // handle the error here
  35.                 return 0
  36.         }
  37.         defer file.Close()
  38.  
  39.         // get the file size
  40.         stat, err := file.Stat()
  41.         if err != nil {
  42.                 return 0
  43.         }
  44.         return uint64(stat.Size())
  45. }
  46.  


OUTPUT:
Source File Size: 3.2 MB
Compressed File Size: 1.2 MB
Extracted File Size: 3.2 MB

real   0m0.248s
user   0m0.224s
sys   0m0.019s


AIR.

Offline jalih

  • Advocate
  • Posts: 75
Re: String Compression and Decompression
« Reply #5 on: March 07, 2019, 09:47:53 AM »
Here is 8th version...

Zipping and unzipping file:

Code: [Select]
: app:main
  "warpeace.txt" f:size nip "Source file size is: %d bytes\n" s:strfmt .
  f:zipnew "warpeace.txt" "warpeace.txt" f:zip+
  "warpeace.zip" f:zipsave

  "warpeace.zip" f:size nip "Compressed file size is: %d bytes\n" s:strfmt .

  "warpeace.zip" f:zipopen getcwd true f:unzip drop
  "warpeace.txt" f:size nip "Extracted file size is: %d bytes\n" s:strfmt .
  bye ;

Read text file into buffer, zip it and then unzip data back to buffer:

Code: [Select]
: app:main
  f:zipnew "warpeace.txt" f:slurp
  b:len "Source buffer size is: %d bytes\n" s:strfmt .
  "warpeace.txt" swap f:zip+
  "warpeace.zip" f:zipsave

  "warpeace.zip" f:size nip "Compressed file size is: %d bytes\n" s:strfmt .

  "warpeace.zip" f:zipopen
  "warpeace.txt" f:zip@ nip b:len nip "Extracted buffer size was: %d bytes\n" s:strfmt .
  bye ;
« Last Edit: March 07, 2019, 09:50:27 AM by jalih »

Offline John

  • Forum Support / SB Dev
  • Posts: 2879
    • ScriptBasic Open Source Project
Re: String Compression and Decompression
« Reply #6 on: March 07, 2019, 06:38:20 PM »
Your avatar says Mac Daddy but I'm seeing GO Daddy:)

Offline AIR

  • BASIC Developer
  • Posts: 741
Re: String Compression and Decompression
« Reply #7 on: March 07, 2019, 08:59:51 PM »
Your avatar says Mac Daddy but I'm seeing GO Daddy:)

I'm coding GO on my MAC.... ;)

Offline AIR

  • BASIC Developer
  • Posts: 741
Re: String Compression and Decompression
« Reply #8 on: March 07, 2019, 09:16:05 PM »
Jalih, thanks for your example, that's pretty cool!

So 8th automatically pipes the output of a command or file to the next command, right?

At first glance it's kind of cryptic, but when I thought about how you can pipe the output of a command to another command in shell scripting or Powershell, your code made perfect sense.

What does the trailing DOT mean, though?  I do like how concise/compact 8th code is, btw!

Without checking the docs, am I correct in thinking that f: is for file related functions and s: is for strings?

AIR.


Offline jalih

  • Advocate
  • Posts: 75
Re: String Compression and Decompression
« Reply #9 on: March 08, 2019, 01:08:14 AM »
So 8th automatically pipes the output of a command or file to the next command, right?

At first glance it's kind of cryptic, but when I thought about how you can pipe the output of a command to another command in shell scripting or Powershell, your code made perfect sense.

What does the trailing DOT mean, though?  I do like how concise/compact 8th code is, btw!

Yes, 8th is a stack based language. You use stack to pass data around. Global variables, word-local and task-local variables are supported but most of the power comes from factoring code into small words, composing them and using stack efficiently.

Dot is a Forth style print word, on my example it just prints the string.

Without checking the docs, am I correct in thinking that f: is for file related functions and s: is for strings?

You are right, those are namespaces and help to write more modular code.
« Last Edit: March 08, 2019, 08:14:56 AM by jalih »