Create dynamic Objects in Go like Javascript (JSON)

Recently, I got a new task to implement a new API in our Go microservice to save the telemetry data (event data) into the new MongoDB instance.

The problem

I realized that the payload of the events does not have a specific schema. Excludes some properties like userId, IP address, and event name, they have a different schema from all the rest elements.

For example, I have 2 events:

{
  "context": {
    "ip": "203.192.213.46",
    "library": {
      "name": "unknown",
      "version": "unknown"
    }
  },
  "properties": {
    "instanceId": "<uuid>"
  },
  "type": "track",
  "userId": "203.192.213.46"
}

{
  "context": {
    "library": {
      "name": "analytics-java",
      "version": "2.1.1"
    }
  },
  "event": "execute_ACTION_TRIGGERED",
  "integrations": {},
  "messageId": "0f6b07ee-0717-413-808c-c25b09c0468",
  "originalTimestamp": "2021-08-24T07:23:35.610Z",
  "properties": {
    "appId": "612465f87b2230debedfc6",
    "appMode": "edit",
    "appName": "APP1",
    "datasource": {
      "name": "Test App"
    },
    "instanceId": "612460418944011a10fa5b",
    "isExampleApp": false,
    "isSuccessfulExecution": true,
    "name": "Test",
    "orgId": "612464f7f230debedfc4",
    "originService": "appsmith-server",
    "pageId": "612465802230debedfc8",
    "pageName": "Page1",
    "pluginName": "PostgreSQL",
    "statusCode": "",
    "timeElapsed": 8,
    "type": "DB",
    "username": "70280e5d07e61e5e915e5d26ac8704bbd68d3f75ebad67ba439f4c354d7"
  },
  "receivedAt": "2021-08-24T07:23:39.996Z",
  "sentAt": "2021-08-24T07:23:39.885Z",
  "timestamp": "2021-08-24T07:23:35.721Z",
  "type": "track",
  "userId": "70280e5dd9e61e5e91526ac8704bbd68d3f75ebad67ba439f4c354d7",
}

In this case, I can’t define a golang struct to define all of its members. Because when we unmarshal the JSON into the struct, we will have many “zero values”, which is not necessary. So I need another type to just forward the JSON received directly into MongoDB.

Solution in Typescript

Javascript Object’s attribute can be dynamically modified, even in Typescript:

type AllowedTypes = string | number | object | Date;

type DynamicEvent = {
    userId: string,
    event: string,
    [K : string]: AllowedTypes
}
const e : DynamicEvent = {
    userId: "userId",
    event: "some-event",
    "properties":{
        ip: "192.168.x.x",
        date: new Date()
    }
}

Solution in Go

In Go, we can’t create the dynamic struct member with dynamic data types, but we have a data structure called map, specifically map[string]interface{} .

package main

import (
	"encoding/json"
	"fmt"
)

type EventLog map[string]interface{}

func main() {
	logMap := EventLog{
		"context": map[string]interface{}{
			"ip": "203.192.213.46",
		},
		"userId": "203.192.213.46",
		"type":   "track",
	}
	req, _ := json.Marshal(logMap)
	fmt.Println("map to json:\n", string(req))

	var b EventLog
	_ = json.Unmarshal(req, &b)

	fmt.Println("Unmarshalled:\n", b)
}

You can run this and get the output:

Reference (Golang dynamically creating member of struct):

https://stackoverflow.com/questions/40559250/golang-dynamically-creating-member-of-struct