CODEWITHSUNDEEP
jsongitbashshellparsing
Bergerova
Bergerova

Reputation: 903

Parse git log file names to json

I use the following code to get the name of the files which were changed in the last commit:

git log -1 --stat

Now I want to parse the results to JSON. I know that I can use pretty-formats to parse all 'git log' data into JSON (pretty-formats) like this:

git log \
--pretty=format:'{%n  "commit": "%H",%n  "author": "%an <%ae>",%n  "date": "%ad",%n  "message": "%f"%n},' \
$@ | \
perl -pe 'BEGIN{print "["}; END{print "]\n"}' | \
perl -pe 's/},]/}]/'

but is there a way to also parse to this JSON the change file name for example?

Upvotes: 4

Views: 1666

Answers (2)

Reino
Reino

Reputation: 3423

Instead of difficult regular expressions and custom Bash functions, you can create the JSON array from just 1 git-command with , a true JSON parser.

To illustrate I'll take the latest 3 commits from the FFmpeg repo.

$ git log -3 --pretty=format:'%H%n%an <%ae>%n%ad%n%f' --name-only
cf12a478b206cd107343827426a05aedb83816bc
Thilo Borgmann <[email protected]>
Sun Jun 6 15:15:50 2021 +0200
fftools-cmdutils.c-Add-cmd-line-option-to-override-detection-of-cpu-count
doc/fftools-common-opts.texi
fftools/cmdutils.c
fftools/cmdutils.h

87951dcbe775b349a671b9ac2e6ac5c38aee0e79
Thilo Borgmann <[email protected]>
Sun Jun 6 15:15:00 2021 +0200
lavu-cpu.c-Add-av_force_cpu_count-to-override-auto-detection
libavutil/cpu.c
libavutil/cpu.h

b7266302a40ba48fea7a5644f08623159b3dcac7
Keyun Tong <[email protected]>
Sun Jun 20 21:42:29 2021 +0200
fftools-ffmpeg-Add-new-variant-source_no_drop-to-the-force_key_frames-option
doc/ffmpeg.texi
fftools/ffmpeg.c
fftools/ffmpeg.h

$ git log -3 --pretty=format:'%H%n%an <%ae>%n%ad%n%f' --name-only | xidel -se '
  array{
    for $cmt in tokenize($raw,"\n\n")
    let $ln:=x:lines($cmt)
    return {
      "commit":$ln[1],
      "author":$ln[2],
      "date":$ln[3],
      "message":$ln[4],
      "files":array{$ln[position() = 5 to last()]}
    }
  }
'
[
  {
    "commit": "cf12a478b206cd107343827426a05aedb83816bc",
    "author": "Thilo Borgmann <[email protected]>",
    "date": "Sun Jun 6 15:15:50 2021 +0200",
    "message": "fftools-cmdutils.c-Add-cmd-line-option-to-override-detection-of-cpu-count",
    "files": ["doc/fftools-common-opts.texi", "fftools/cmdutils.c", "fftools/cmdutils.h"]
  },
  {
    "commit": "87951dcbe775b349a671b9ac2e6ac5c38aee0e79",
    "author": "Thilo Borgmann <[email protected]>",
    "date": "Sun Jun 6 15:15:00 2021 +0200",
    "message": "lavu-cpu.c-Add-av_force_cpu_count-to-override-auto-detection",
    "files": ["libavutil/cpu.c", "libavutil/cpu.h"]
  },
  {
    "commit": "b7266302a40ba48fea7a5644f08623159b3dcac7",
    "author": "Keyun Tong <[email protected]>",
    "date": "Sun Jun 20 21:42:29 2021 +0200",
    "message": "fftools-ffmpeg-Add-new-variant-source_no_drop-to-the-force_key_frames-option",
    "files": ["doc/ffmpeg.texi", "fftools/ffmpeg.c", "fftools/ffmpeg.h"]
  }
]
  • Create a sequence of every commit by "tokenizing" on the empty line.
  • For every commit create a variable that holds a sequence of every line.
    (x:lines($cmt) is a shorthand for tokenize($cmt,'\r\n?|\n'))
  • Return the JSON object from each corresponding line, where the list of files becomes an array from line nr.5 to the last one.

Upvotes: 2

Bertrand Martel
Bertrand Martel

Reputation: 45372

You can do the following :

function getcommit { \
    git show --pretty="format:"  --name-only $1 | \
    perl -pe's/^\n//g;' | \
    sed 's/\(.*\)/"\1"/g' | \
    perl -0pe 's/\n(?!\Z)/,\n/g'; \
}

export -f getcommit

git log -1 --pretty=format:'{%n  "commit": "%H",%n  "author": "%an <%ae>",%n  "date": "%ad",%n  "message": "%f",%n  "files": [ COMMIT_HASH_%H  ]%n},' | \
perl -pe 'BEGIN{print "["}; END{print "]\n"}' | \
perl -pe 's/},]/}]/;s/COMMIT_HASH_(\w+)/`echo -n "";getcommit $1`/e'

Basically, I replaced commit hash with a fixed string COMMIT_HASH_ preceding the hash itself and then replace this hash with the result of git show --pretty="format:" --name-only $COMMIT_HASH.

All changed file are put into a json array "files". This is working for the last X commit

Here is an example for the 2 last commit :

[{
  "commit": "1edcef90b42afee11fbd31dcc458ae0f15a3bb6e",
  "author": "Bertrand Martel <[email protected]>",
  "date": "Tue Oct 13 17:35:34 2015 +0200",
  "message": "update-readme",
  "files": [ "README.md",
"device.png",
"screenshot.png"
  ]
},
{
  "commit": "8aa2ce64e58b770122a3561b8ef41d807ce36abc",
  "author": "Bertrand Martel <[email protected]>",
  "date": "Mon Oct 12 19:36:18 2015 +0200",
  "message": "fix-async-bluetooth-command-bug-bluetoooth-state-check",
  "files": [ "android/app/src/main/java/fr/bmartel/android/bluetooth/BluetoothCustomManager.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/GattTask.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/GattUtils.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/IBluetoothCustomManager.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/IBluetoothManagerEventListener.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/ICharacteristicListener.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/IDevice.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/IDeviceInitListener.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/IScanListListener.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/connection/BluetoothDeviceAbstr.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/connection/BluetoothDeviceConn.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/connection/IBluetoothDeviceConn.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/listener/IPushListener.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/notti/INottiDevice.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/notti/INottiListener.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/notti/NottiDevice.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/shared/ActionFilterGatt.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/shared/ISharedActivity.java",
"android/app/src/main/java/fr/bmartel/android/bluetooth/shared/LeDeviceListAdapter.java",
"android/app/src/main/java/fr/bmartel/android/notti/NottiActivity.java",
"android/app/src/main/java/fr/bmartel/android/notti/NottiBtService.java",
"android/app/src/main/java/fr/bmartel/android/notti/NottiDeviceActivity.java"
  ]
}]

Here is a script that take commit index in parameters and return json info including files changed : https://gist.github.com/bertrandmartel/a4ed5d76562e74d77282

Upvotes: 2

Related Questions