Author Topic: JSON Challenge  (Read 1620 times)

Offline AIR

  • BASIC Developer
  • Posts: 684
JSON Challenge
« on: November 24, 2018, 04:22:16 PM »
Time for a new challenge:

Using the attached json file, which was pulled from github and shows commit status, show how your language of choice would extract the following fields:

  • Name of Commiter
  • Date of Commit
  • The SHA HASH for that Commit

Using Libraries/Built in JSON support is allowed; I thought about restricting this to only "roll your own" implementations, but wanted to make this challenge accessible to a wider audience, since coding a home-made parser might be beyond the reach of most, especially time-wise.

AIR.

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: JSON Challenge
« Reply #1 on: November 24, 2018, 04:44:19 PM »
I'm going to try native SB syntax to create a BASIC only extension json module. Using an external library for something based on a limited definition seems like overkill to me. My SB XML parser was less than 10 lines.

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: JSON Challenge
« Reply #2 on: November 24, 2018, 06:12:04 PM »
I'm going to try native SB syntax to create a BASIC only extension json module. Using an external library for something based on a limited definition seems like overkill to me. My SB XML parser was less than 10 lines.

Where simple text parsers become not so simple is when you have to handle deeply nested "objects", malformed data sets, and validation of those sets. Also you may need to account for UTF-8.

JSON is not just about brackets; there's a bunch of other things that the official spec supports.

With all that said, go for it! 

AIR.

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: JSON Challenge
« Reply #3 on: November 24, 2018, 06:26:00 PM »
One of my personal goals with this challenge is to implement an interface that makes it easy to grab the data you need.

For me, it's not only about can the challenge be met, but also about how simple it is for a programmer to use what you've created for other purposes.

With that said, he's what I worked up this evening, using HOTBASIC:

Code: Text
  1. '-------------------------------------------------------
  2. ' DATE: 11/24/2018
  3. ' NAME: parse.bas
  4. ' DESCRIPTION: JSON Parsing demo, using libparson
  5. ' AUTHOR:  Armando I. Rivera (AIR)
  6. ' LANGUAGE:  HOTBASIC! v7.1
  7. ' EDITOR:         AIRSyS-IDE
  8. '-------------------------------------------------------
  9.  
  10. $apptype console: $typecheck on
  11.  
  12. $include "JSON.bas"
  13.  
  14. deflng item, i
  15. dim json as JSON
  16.  
  17. json.init("commits.json")
  18.  
  19. if json.type = JSONArray then
  20.         for i = 0 to json.count-1
  21.                 item = json.object(i)
  22.                 print string$(50,"-")
  23.                 print "Name: "; json.text(item,"commit.author.name")
  24.                 print "Commit Date: "; json.text(item, "commit.author.date")
  25.                 print "SHA Hash: "; json.text(item, "sha")
  26.         next
  27.         print string$(50,"-")
  28. end if
  29.  
  30. end
  31.  

The JSON.bas file consists of:
Code: Text
  1. '-------------------------------------------------------
  2. '   NAME:                       JSON.bas
  3. '   Version:            1.0
  4. '   DATE:                       2018-11-24
  5. '-------------------------------------------------------
  6. ' DEVELOPMENT ENVIRONMENT
  7. '   OpSys:                      Windows
  8. '   Language:   HotBasic
  9. '   Editor:                     AIRSyS-IDE
  10. '   Libraries:          libparson.dll
  11. '-------------------------------------------------------
  12. ' AUTHOR:               Armando I. Rivera (AIR)
  13. '-------------------------------------------------------
  14. ' DESCRIPTION:
  15. '                                               libparson Json parser interface
  16. '
  17. '-------------------------------------------------------
  18. ' Terms of Use/Licence
  19. '                                               MIT
  20. '-------------------------------------------------------
  21.  
  22. declare sub JSONinit (filename as string)
  23. declare function JSONtext(json_object as long,query as string) as string
  24. declare function JSONobject(index as long) as long
  25.  
  26. OBJECT JSON
  27.         root as long
  28.         items as long
  29.         count as long
  30.         type as long
  31.         text as Function
  32.         object as function
  33.         init as Sub
  34. END OBJECT
  35.  
  36. ' JSON VALUE TYPE CODES
  37. CONST    JSONError      =       -1
  38. CONST    JSONNull               =       1
  39. CONST    JSONString     =       2
  40. CONST    JSONNumber     =       3
  41. CONST    JSONObject     =       4
  42. CONST    JSONArray      =       5
  43. CONST    JSONBoolean     =      6
  44.  
  45. begin runonce
  46.         cdecl function json_parse_file lib "libparson.dll"   (filename as string) as long
  47.         cdecl function json_parse_string lib "libparson.dll" (strBuffer as string) as long
  48.         cdecl function json_value_get_type  lib "libparson.dll"   (json_object as long) as long
  49.         cdecl function json_value_get_array  lib "libparson.dll"   (json_object as long) as long
  50.         cdecl function json_array_get_count  lib "libparson.dll"   (json_object as long) as long
  51.         cdecl function json_object_get_string lib "libparson.dll" (json_object as long, keyname as string) as string
  52.         cdecl function json_object_dotget_string lib "libparson.dll"  (json_object as long, keypath as string) as string
  53.         cdecl function json_array_get_object lib "libparson.dll" (json_object as long, index as long) as long
  54.         cdecl function json_object_dotget_value lib "libparson.dll" (json_object as long, keypath as string) as long
  55.         cdecl function json_object_get_value lib "libparson.dll" (json_object as long, keypath as string) as long
  56. end runonce
  57.  
  58. Sub JSONinit(filename as string)
  59.         self.root = json_parse_file(filename)
  60.         self.type = json_value_get_type(self.root)
  61.         if self.type = JSONArray then
  62.                 self.items = json_value_get_array(self.root)
  63.                 self.count = json_array_get_count(self.items)
  64.         end if
  65. End Sub
  66.  
  67. function JSONtext(json_object as long, query as string) as string
  68.         defstr res
  69.        
  70.         if instr(query, ".") then
  71.                 if  json_object_dotget_value(json_object,query) <> 0 then
  72.                         res = json_object_dotget_string(json_object, query)
  73.                 end if
  74. '               print res
  75.         else
  76.                 if  json_object_get_value(json_object,query) <> 0 then
  77.                         res = json_object_get_string(json_object, query)
  78.                 end if
  79.         end if
  80.        
  81.         result = res
  82. end function
  83.  
  84. function JSONobject(index as long) as long
  85.         result =  json_array_get_object(self.items, index)
  86. end function
  87.  
  88.  

Attached you find the output, and the 32bit dll (HOTBASIC is 32bit only)

AIR.
« Last Edit: November 24, 2018, 10:59:36 PM by AIR »

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: JSON Challenge
« Reply #4 on: November 24, 2018, 06:45:47 PM »
Script BASIC

Code: Script BASIC
  1. OPEN "commits.json" FOR INPUT AS #1
  2. fl = FILELEN("commits.json")
  3. json = INPUT(fl,1)
  4. SPLITA json BY _
  5. """  {
  6.    "sha": """ _
  7. TO commits
  8. FOR i = 1 TO UBOUND(commits)
  9.   IF commits[i] LIKE """*committer*"name": "*"*date": "*"*""" THEN
  10.    PRINT JOKER(3),"\n"
  11.    PRINT JOKER(5),"\n"
  12.    PRINT MID(commits[i],2,40),"\n"
  13.    PRINT STRING(40,"="),"\n"
  14.  END IF
  15. NEXT
  16.  


jrs@jrs-laptop:~/sb/abcc/json$ time scriba jsoncc.sb
Linus Torvalds
2018-11-24T20:58:47Z
e195ca6cb6f21633e56322d5aa11ed59cdb22fb2
========================================
Linus Torvalds
2018-11-24T17:42:32Z
d146194f31c96f9b260c5a1cf1592d2e7f82a2e2
========================================
Linus Torvalds
2018-11-24T17:19:38Z
857fa628bbe93017c72ddd0d5304962a2608db07
========================================
Linus Torvalds
2018-11-24T17:11:52Z
abe72ff4134028ff2189d29629c40a40bee0a989
========================================
David S. Miller
2018-11-24T06:35:38Z
07093b76476903f820d83d56c3040e656fb4d9e3
========================================
David S. Miller
2018-11-24T06:34:40Z
3fa528b7682e73e906266bcd43728b8f923bf9b2
========================================
David S. Miller
2018-11-24T06:33:55Z
e7b9fb4f545b1f7885e7c642643828f93d3d79c9
========================================
David S. Miller
2018-11-24T06:31:56Z
ef2a7cf1d8831535b8991459567b385661eb4a36
========================================
David S. Miller
2018-11-24T01:24:24Z
c44c749d3b6fdfca39002e7e48e03fe9f9fe37a3
========================================
David S. Miller
2018-11-24T01:18:15Z
5ed9dc99107144f83b6c1bb52a69b58875baf540
========================================
David S. Miller
2018-11-23T19:59:40Z
18ba58e1c234ea1a2d9835ac8c1735d965ce4640
========================================
David S. Miller
2018-11-23T19:59:40Z
e59ff2c49ae16e1d179de679aca81405829aee6c
========================================
Linus Torvalds
2018-11-23T19:24:55Z
7c98a42618271210c60b79128b220107d35938d9
========================================
Linus Torvalds
2018-11-23T19:20:14Z
3381918fec9278d14f776d1dabd68da85fd6822e
========================================
David S. Miller
2018-11-23T19:20:02Z
484afd1bd3fc6f9f5347289fc8b285aa65f67054
========================================
David S. Miller
2018-11-23T19:18:53Z
605108acfe6233b72e2f803aa1cb59a2af3001ca
========================================
David S. Miller
2018-11-23T19:17:56Z
896585d48e8e9ba44cd1754fbce8537feffcc1a5
========================================
Linus Torvalds
2018-11-23T19:15:27Z
d88783b9c8849d88c3a75b7b9071cba072b47eba
========================================
David S. Miller
2018-11-23T19:08:03Z
5cd8d46ea1562be80063f53c7c6a5f40224de623
========================================
Linus Torvalds
2018-11-23T18:56:16Z
a03bac580ae743d5900af626ac63f7f8cd85def9
========================================
Linus Torvalds
2018-11-23T18:52:57Z
b88af994872418f0a98db6f4a9bae849315a99b0
========================================
Catalin Marinas
2018-11-23T18:44:16Z
4f9f49646a5733c0c2bd49940673dde89a9c5add
========================================
Linus Torvalds
2018-11-23T18:40:19Z
e6005d3c42336074de3745718ac85807dd6e1e6a
========================================
Linus Torvalds
2018-11-23T18:36:02Z
dcd3aa31dcdd6d8eae8d4771c44aeb3b1fec995a
========================================
Linus Torvalds
2018-11-23T18:03:08Z
9b7c880c834c0a1c80a1dc6b8a0b19155361321f
========================================
Catalin Marinas
2018-11-23T17:33:27Z
b5d9a07ef7736b2456b9d3c90568de25e43d8ec3
========================================
Rafael J. Wysocki
2018-11-23T09:32:49Z
1d50088ca3956e5dcd2751a658e7869b9af10bb4
========================================
Rafael J. Wysocki
2018-11-23T09:32:22Z
bec00cb5e97c19d2c8bd10db15d334cb40760000
========================================
Dave Airlie
2018-11-23T01:03:21Z
98c9cdfd34fbb62886e4c5a07e33661aa3352ef5
========================================
David S. Miller
2018-11-22T19:53:26Z
039e70a70c8417b5bf5878a60612ebd2c95f731e
========================================

real   0m0.027s
user   0m0.015s
sys   0m0.011s
jrs@jrs-laptop:~/sb/abcc/json$


« Last Edit: November 24, 2018, 08:41:05 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: JSON Challenge
« Reply #5 on: November 24, 2018, 07:10:19 PM »
Very nice, but you're pulling the wrong SHA hash.

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: JSON Challenge
« Reply #6 on: November 24, 2018, 08:19:47 PM »
Fixed!

I changed my SPLITA pattern which reduced the number of LIKE checks in half making it a bit faster. I think I got the right SHA hash you were looking for.
« Last Edit: November 24, 2018, 09:24:04 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: JSON Challenge
« Reply #7 on: November 24, 2018, 08:47:33 PM »
Yep, that does it all right!

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: JSON Challenge
« Reply #8 on: November 24, 2018, 08:51:41 PM »
Maybe for extra points to make this challenging you could ask for a json generator from a SQLite DB. Then I could show off SB's external pre-processor tricks.
« Last Edit: November 24, 2018, 08:53:15 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: JSON Challenge
« Reply #9 on: November 24, 2018, 08:53:29 PM »
I'm waiting on jalih to chime in.   ;D ;D

Here's a POWERSHELL version:

Code: PowerShell
  1. $jsondata =  Get-Content -Raw -Path .\commits.json | ConvertFrom-Json
  2. $jsondata | % {$_.commit.author.name; $_.commit.author.date; $_.sha; ('-'*40)}

Linus Torvalds
2018-11-24T20:58:47Z
e195ca6cb6f21633e56322d5aa11ed59cdb22fb2
----------------------------------------
Linus Torvalds
2018-11-24T17:42:32Z
d146194f31c96f9b260c5a1cf1592d2e7f82a2e2
----------------------------------------
Linus Torvalds
2018-11-24T17:19:38Z
857fa628bbe93017c72ddd0d5304962a2608db07
----------------------------------------
Linus Torvalds
2018-11-24T17:11:52Z
abe72ff4134028ff2189d29629c40a40bee0a989
----------------------------------------
Andreas Fiedler
2018-11-23T23:16:34Z
07093b76476903f820d83d56c3040e656fb4d9e3
----------------------------------------
Quentin Schulz
2018-11-23T18:01:51Z
3fa528b7682e73e906266bcd43728b8f923bf9b2
----------------------------------------
Fabio Estevam
2018-11-23T17:46:50Z
e7b9fb4f545b1f7885e7c642643828f93d3d79c9
----------------------------------------
Lorenzo Bianconi
2018-11-23T17:28:01Z
ef2a7cf1d8831535b8991459567b385661eb4a36
----------------------------------------
Yangtao Li
2018-11-22T12:34:41Z
c44c749d3b6fdfca39002e7e48e03fe9f9fe37a3
----------------------------------------
Hangbin Liu
2018-11-22T08:15:28Z
5ed9dc99107144f83b6c1bb52a69b58875baf540
----------------------------------------
Jason Wang
2018-11-22T06:36:31Z
18ba58e1c234ea1a2d9835ac8c1735d965ce4640
----------------------------------------
Jason Wang
2018-11-22T06:36:30Z
e59ff2c49ae16e1d179de679aca81405829aee6c
----------------------------------------
Linus Torvalds
2018-11-23T19:24:55Z
7c98a42618271210c60b79128b220107d35938d9
----------------------------------------
Linus Torvalds
2018-11-23T19:20:14Z
3381918fec9278d14f776d1dabd68da85fd6822e
----------------------------------------
Davide Caratti
2018-11-21T17:23:53Z
484afd1bd3fc6f9f5347289fc8b285aa65f67054
----------------------------------------
Paolo Abeni
2018-11-21T17:21:35Z
605108acfe6233b72e2f803aa1cb59a2af3001ca
----------------------------------------
Hangbin Liu
2018-11-21T13:52:33Z
896585d48e8e9ba44cd1754fbce8537feffcc1a5
----------------------------------------
Linus Torvalds
2018-11-23T19:15:27Z
d88783b9c8849d88c3a75b7b9071cba072b47eba
----------------------------------------
Willem de Bruijn
2018-11-20T18:00:18Z
5cd8d46ea1562be80063f53c7c6a5f40224de623
----------------------------------------
Linus Torvalds
2018-11-23T18:56:16Z
a03bac580ae743d5900af626ac63f7f8cd85def9
----------------------------------------
Linus Torvalds
2018-11-23T18:52:57Z
b88af994872418f0a98db6f4a9bae849315a99b0
----------------------------------------
Will Deacon
2018-11-21T15:07:00Z
4f9f49646a5733c0c2bd49940673dde89a9c5add
----------------------------------------
Linus Torvalds
2018-11-23T18:40:19Z
e6005d3c42336074de3745718ac85807dd6e1e6a
----------------------------------------
Linus Torvalds
2018-11-23T18:36:02Z
dcd3aa31dcdd6d8eae8d4771c44aeb3b1fec995a
----------------------------------------
Linus Torvalds
2018-11-23T18:03:08Z
9b7c880c834c0a1c80a1dc6b8a0b19155361321f
----------------------------------------
Sergey Matyukevich
2018-11-16T18:21:30Z
b5d9a07ef7736b2456b9d3c90568de25e43d8ec3
----------------------------------------
Rafael J. Wysocki
2018-11-23T09:32:49Z
1d50088ca3956e5dcd2751a658e7869b9af10bb4
----------------------------------------
Rafael J. Wysocki
2018-11-23T09:32:22Z
bec00cb5e97c19d2c8bd10db15d334cb40760000
----------------------------------------
Dave Airlie
2018-11-23T01:03:20Z
98c9cdfd34fbb62886e4c5a07e33661aa3352ef5
----------------------------------------
David S. Miller
2018-11-22T19:53:26Z
039e70a70c8417b5bf5878a60612ebd2c95f731e
----------------------------------------



AIR.
« Last Edit: November 24, 2018, 08:58:31 PM by AIR »

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: JSON Challenge
« Reply #10 on: November 24, 2018, 09:33:19 PM »
Wow!

A double trailer semi. (PowerHell & JavaShit)  :(

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: JSON Challenge
« Reply #11 on: November 24, 2018, 10:11:29 PM »
SB.js

Script BASIC does JavaShit as well.

Code: Script BASIC
  1. IMPORT js.inc
  2.  
  3. jscode = """
  4. JSON.stringify( { 'foo':1, 'bar':2, 'baz':{ 'quux':3 } }, null, '\t');
  5. """
  6.  
  7. JS::CREATE
  8. PRINT JS::EXEC(jscode),"\n"
  9. JS::DESTROY
  10.  


jrs@jrs-laptop:~/sb/examples/js$ time scriba js_json.sb
{"baz":{"quux":3},"bar":2,"foo":1}

real   0m0.034s
user   0m0.030s
sys   0m0.004s
jrs@jrs-laptop:~/sb/examples/js$

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: JSON Challenge
« Reply #12 on: November 24, 2018, 11:05:47 PM »
Couple of things regarding the SB submission:

1) You're keying on "committer", which can be different from the person who actually authored the code.

2) If you wanted to filter on say the logon of the person who performed the commit, you would have to alter/add your like statement.

3) If you wanted to add, say, the commit message you'd have to alter/add to your like statement.

While your submission gets the data as I originally set in the challenge, your solution has an issue with flexibility.  If, for example, the source file changes, you have to go back and redo your lookup code.

With my HB submission, altering the code to do steps 2/3 above is as simple as:

Code: Text
  1.                 if json.text(item, "author.login") = "torvalds" then
  2.                         print string$(50,"-")
  3.                         print "Name: "; json.text(item,"commit.author.name")
  4.                         print "Commit Date: "; json.text(item, "commit.author.date")
  5.                         print "SHA Hash: "; json.text(item, "sha")
  6.                         print "Commit Message: "; json.text(item,"commit.message")
  7.                 end if

I didn't have to modify the MODULE code, I just added the filter and the commit message to the main program.

The same thing with PowerShell:
Code: PowerShell
  1. $jsondata =  Get-Content -Raw -Path .\commits.json | ConvertFrom-Json
  2. $jsondata | % {
  3.     if ($_.author.login -eq "torvalds") {
  4.         $_.commit.author.name; $_.commit.author.date; $_.sha; $_.commit.message;('-'*40);
  5.     }
  6. }
  7.  

Note that I'm not putting down your submission, I'm just pointing out some of the issues that can arise when you do a manual text extraction.

AIR.

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: JSON Challenge
« Reply #13 on: November 24, 2018, 11:07:50 PM »
SB.js

Script BASIC does JavaShit as well.

Code: Script BASIC
  1. IMPORT js.inc
  2.  
  3. jscode = """
  4. JSON.stringify( { 'foo':1, 'bar':2, 'baz':{ 'quux':3 } }, null, '\t');
  5. """
  6.  
  7. JS::CREATE
  8. PRINT JS::EXEC(jscode),"\n"
  9. JS::DESTROY
  10.  


jrs@jrs-laptop:~/sb/examples/js$ time scriba js_json.sb
{"baz":{"quux":3},"bar":2,"foo":1}

real   0m0.034s
user   0m0.030s
sys   0m0.004s
jrs@jrs-laptop:~/sb/examples/js$


Then you should have no problem using js to properly parse json.

AIR.

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: JSON Challenge
« Reply #14 on: November 25, 2018, 01:32:23 AM »
Quote from: AIR
Name of Commiter

The challenge is very specific. Read a json file and extract the 3 fields you specified. My script does just that. It is not a generic json library routine. I really don't think maintaining the code isn't very difficult to return the data you're looking for.

Quote
Then you should have no problem using js to properly parse json.

The JavaScript library I'm using for the extension module is no longer maintained and deprecated for a direction I have no interest in. It doesn't work correctly on Windows. It's just my personal JavaScript toy for Linux.
« Last Edit: November 25, 2018, 02:01:51 PM by John »