überSpark Manifest

überSpark uses a unified manifest interface for specifying and configuring various framework options.

A überSpark manifest uses the JavaScript Object Notation (JSON) text format and is identified by the file name uberspark.json

The überSpark JSON manifest is essentially a collection of name/value pairs and/or ordered list of values. See the JSON official specification (https://json.org) for more details on the JSON format.

Note

We chose JSON for its lightweight nature that allows reading and writing of options with ease. JSON also has a formal specification of its schema which is turn enables creation of correct-by-construction parsers and generators for runtime use.

A überSpark manifest (uberspark.json) has the following general high-level structure:

{
    /* user-defined block comments can appear anywhere */
    ...
    // user-defined single line comments can appear anywhere
    ...
    "manifest_node" : {
        ...
    },

    ...

    "manifest_node" : {

    }
}

Here manifest_node can be one or more of the following specific node definitions as described below.

Note

The type and combination of specific node definitions will vary based on the functionality the manifest is describing. For example, the manifest describing a üobject will have a uberspark-uobj node, whereas a manifest describing a üobject collection will not.

uberspark-manifesthdr Manifest Node

Every überSpark manifest needs to define a uberspark-manifesthdr node at the bare minimum, which describes the types of manifest nodes contained in the manifest along with the überSpark framework version requirements.

The JSON declaration of the uberspark-manifesthdr node is as below:

uberspark-manifesthdr
Object Properties:
 
  • manifest_node_type (string list) – comma seperated list of zero or more manifest node types (as strings) that are part of the manifest (“uberspark-uobj”, “uberspark-uobjcoll”, “uberspark-config”, “uberspark-sentinel”)
  • uberspark_min_version (string) – minimum <version> of überSpark required (“<version>”, “any”)
  • uberspark_max_version (string) – maximum <version> of überSpark supported (“<version>”, “any”)

An example definition of the uberspark-manifesthdr node within uberspark.json follows:

{
    "uberspark-manifest":{
        "manifest_node_types" : [ "uberspark-uobjcoll", "uberspark-config" ],
        "uberspark_min_version" : "6.0.0",
        "uberspark_max_version" : "any"
    }
}

uberspark-config Manifest Node

The uberspark-config node within the überSpark manifest can be (optionally) used to selectively override configuration settings within the current staging environment.

Note

You can define only the required node properties that you want to override within the uberspark-config node definition. See example that follows below.

The JSON declaration of the uberspark-config node is as below:

uberspark-config
Object Properties:
 
  • binary_page_size (string) – size in bytes of a memory page within the üobject/üobject collection binary (“<hexadecimalinteger>”, default=”0x200000”)
  • binary_uobj_default_section_size (string) – size in bytes of a üobject binary section (e.g., code, data) (“<hexadecimalinteger>”, default=”0x200000”)
  • binary_uobj_section_alignment (string) – memory alignment in bytes, of üobject binary section (“<hexadecimalinteger>”, default=”0x200000”)
  • uobj_binary_image_load_address (string) – memory load address of üobject binary (“<hexadecimalinteger>”, default=”0x200000”)
  • uobj_binary_image_uniform_size (boolean) – indicates if all üobjects within a üobject collection binary have the same size (true, false)
  • uobj_binary_image_size (string) – size in bytes of a üobject binary (“<hexadecimalinteger>”, default=”0x2400000”)
  • uobj_binary_image_alignment (string) – memory alignment in bytes of a üobject binary image (“<hexadecimalinteger>”, default=”0x200000”)
  • uobjcoll_binary_image_load_address (string) – memory load address of üobject collection binary (“<hexadecimalinteger>”, default=”0x60000000”)
  • uobjcoll_binary_image_hdr_section_alignment (string) – memory alignment in bytes, of üobject collection binary header section (“<hexadecimalinteger>”, default=”0x200000”)
  • uobjcoll_binary_image_hdr_section_size (string) – size in bytes of a üobject collection binary header section (“<hexadecimalinteger>”, default=”0x200000”)
  • uobjcoll_binary_image_section_alignment (string) – memory alignment in bytes, of üobject collection binary section (“<hexadecimalinteger>”, default=”0x200000”)
  • bridge_cc_bridge (string) – C compiler <bridge namespace path> (“<bridgenamespacepath>”)
  • bridge_as_bridge (string) – Assembler <bridge namespace path> (“<bridgenamespacepath>”)
  • bridge_ld_bridge (string) – Linker <bridge namespace path> (“<bridgenamespacepath>”)

An example definition of the uberspark-config node within uberspark.json follows:

{
    "uberspark-manifest":{
        "manifest_node_types" : [ "uberspark-config" ],
        "uberspark_min_version" : "any",
        "uberspark_max_version" : "any"
    },

    "uberspark-config":{
            "binary_uobj_section_alignment" : "0x200000",
            "bridge_cc_bridge" : "container/amd64/x86_32/generic/gcc/v5.4.0"
    }
}

The aforementioned definition selectively overrides the binary_uobj_section_alignment and bridge_cc_bridge configuration settings within the current staging environment.

Manifest Nodes for Bridges

überSpark supports a variety of bridges such as compilers, assemblers, linkers, verification tools, build system etc. A supported bridge is described by a bridge manifest node which follows the general layout given below:

{
    "uberspark-bridge-<bridgename>" : {

        //common bridge header definition
        "bridge-hdr":{
            ...
        },

        //bridge specific properties
        ...
    }
}

The JSON declaration of the common bridge-hdr sub-node is as below:

bridge-hdr
Object Properties:
 
  • btype (string) – bridge type (“container”, “native”)
  • bname (string) – <bridge-name> as identified by überSpark (“<brige-name>”)
  • execname (string) – <executable-name> that carries out the functionality of the bridge (“<executable-name>”)
  • devenv (string) – development environment where the bridge executable runs
  • devenv – development environment where the bridge executable runs (“amd64”)
  • arch (string) – CPU architecture supported by the bridge (“x86_32”)
  • cpu (string) – CPU model supported by the bridge (“generic”)
  • version (string) – <version> of the bridge (executable) (“<version>”)
  • path (string) – filesystem <path> to the bridge executable (ignored if btype is “container”) (“<path>”)
  • params (string list) – comma delimited (optional) string options for the bridge executable
  • container_fname (string) – <filename> of the container dockerfile (ignored if btype is “native”) (“<filename>”)

The follow sections describe the bridge specific manifest node declarations for supported bridges.

uberspark-bridge-as Manifest Node

The uberspark-bridge-as node within the überSpark manifest is used to describe an Assembler bridge. The JSON declaration of the uberspark-bridge-as node is as below:

uberspark-bridge-as
Object Properties:
 
  • bridge-hdr (bridge-hdr) – the common bridge header declaration
  • params_prefix_obj (string) – command line option prefix to generate object file from Assembly source
  • params_prefix_output (string) – command line option prefix to specify output file name
  • params_prefix_include (string) – command line option prefix to include a header file

An example definition of the uberspark-bridge-as node for the GNU-as Assembler, within uberspark.json follows:

{
    "uberspark-manifest":{
        "manifest_node_types" : [ "uberspark-bridge-as" ],
        "uberspark_min_version" : "any",
        "uberspark_max_version" : "any"
    },

    "uberspark-bridge-as":{
        "bridge-hdr":{
            "btype" : "container",
            "bname" : "gnu-as",
            "execname" : "gcc",
            "devenv" : "amd64",
            "arch" : "x86_32",
            "cpu" : "generic",
            "version" : "v2.26.1",
            "path" : ".",
            "params" : [ "-m32" ],
            "container_fname" : "uberspark_bridges.Dockerfile"
        },

        "params_prefix_obj" : "-c",
        "params_prefix_output" : "-o",
        "params_prefix_include" : "-I"
    }
}

Note

Here the Assembler bridge type is defined to be a container and uberspark_bridges.Dockerfile is the container dockerfile that includes the build for running GNU-as within an amd64 environment (e.g., ubuntu or alpine)

uberspark-bridge-cc Manifest Node

The uberspark-bridge-cc node within the überSpark manifest is used to describe a C compiler bridge. The JSON declaration of the uberspark-bridge-cc node is as below:

uberspark-bridge-cc
Object Properties:
 
  • bridge-hdr (bridge-hdr) – the common bridge header declaration
  • params_prefix_obj (string) – command line option prefix to generate object file from C source
  • params_prefix_asm (string) – command line option prefix to generate Assembly source from C source
  • params_prefix_output (string) – command line option prefix to specify output file name
  • params_prefix_include (string) – command line option prefix to include a header file

An example definition of the uberspark-bridge-cc node for the GNU gcc C compiler, within uberspark.json follows:

{
    "uberspark-manifest":{
        "manifest_node_types" : [ "uberspark-bridge-cc" ],
        "uberspark_min_version" : "any",
        "uberspark_max_version" : "any"
    },

    "uberspark-bridge-cc":{
        "bridge-hdr":{
            "btype" : "container",
            "bname" : "gcc",
            "execname" : "gcc",
            "devenv" : "amd64",
            "arch" : "x86_32",
            "cpu" : "generic",
            "version" : "v5.4.0",
            "path" : ".",
            "params" : [ "-m32" ],
            "container_fname" : "uberspark_bridges.Dockerfile"
        },

        "params_prefix_obj" : "-c",
        "params_prefix_asm" : "-S",
        "params_prefix_output" : "-o",
        "params_prefix_include" : "-I"
    }
}

Note

Here the C compiler bridge type is defined to be a container and uberspark_bridges.Dockerfile is the container dockerfile that includes the build for running GNU gcc within an amd64 environment (e.g., ubuntu or alpine)

uberspark-bridge-ld Manifest Node

The uberspark-bridge-ld node within the überSpark manifest is used to describe a Linker bridge. The JSON declaration of the uberspark-bridge-ld node is as below:

uberspark-bridge-ld
Object Properties:
 
  • bridge-hdr – the common bridge header declaration
  • params_prefix_lscript (string) – command line option prefix to specify input linker script file
  • params_prefix_libdir (string) – command line option prefix to specify library directory
  • params_prefix_lib (string) – command line option prefix to include a library
  • params_prefix_output (string) – command line option prefix to specify output file name

An example definition of the uberspark-bridge-ld node for the GNU ld linker, within uberspark.json follows:

{
    "uberspark-manifest":{
        "manifest_node_types" : [ "uberspark-bridge-ld" ],
        "uberspark_min_version" : "any",
        "uberspark_max_version" : "any"
    },

    "uberspark-bridge-ld":{
        "bridge-hdr":{
            "btype" : "container",
            "bname" : "gnu-ld",
            "execname" : "ld",
            "devenv" : "amd64",
            "arch" : "x86_32",
            "cpu" : "generic",
            "version" : "v2.26.1",
            "path" : ".",
            "params" : [ "-m elf_i386", "--oformat=elf32-i386"  ],
            "container_fname" : "uberspark_bridges.Dockerfile"
        },

        "params_prefix_lscript" : "-T",
        "params_prefix_libdir" : "-L",
        "params_prefix_lib" : "-l",
        "params_prefix_output" : "-o"
    }
}

Note

Here the Linker bridge type is defined to be a container and uberspark_bridges.Dockerfile is the container dockerfile that includes the build for running GNU ld within an amd64 environment (e.g., ubuntu or alpine) and producing a 32-bit ELF binary

uberspark-uobj Manifest Node

The uberspark-uobj node within the überSpark manifest is used to describe a üobject. The JSON declaration of the uberspark-uobj node is as below:

uberspark-uobj
Object Properties:
 
  • namespace (string) – üobject namespace path as identified by überSpark
  • arch (string) – üobject target hardware platform
  • arch – üobject target CPU architecture (“x86_32”)
  • cpu (string) – üobject target CPU model (“generic”)
  • sources (sources) – üobject sources definition sub-node
  • publicmethods ({}) – comma delimited üobject public methods declarations of the format: “<publicmethod-name>” : [ “<publicmethod-return-type>”, “<publicmethod-params-decl>”, “<publicmethods-numberof-params”>]
  • intrauobjcoll-callees ({}) – comma delimited declarations of public methods of other üobjects that this üobject invokes within the same üobject collection. The declarations are of the format: “<uobj-namespace-path>” : [ “<publicmethod-1-name>”, …, “<publicmethod-n-name>”]
  • interuobjcoll-callees ({}) – comma delimited declarations of public methods of other üobjects that this üobject invokes across üobject collection. The declarations are of the format: “<uobjcoll-namespace-path>” : [ “<publicmethod-1-name>”, …, “<publicmethod-n-name>”]
  • legacy-callees (string list) – comma delimited legacy CoSS function names that this üobject invokes
sources
Object Properties:
 
  • h-files (string list) – comma delimited list of üobject header files
  • c-files (string list) – comma delimited list of üobject C source files
  • casm-files (string list) – comma delimited list of üobject CASM source files
  • asm-files (string list) – comma delimited list of üobject Assembly source files

An example definition of the uberspark-uobj node for a sample üobject called add, within uberspark.json follows:

{
    "uberspark-manifest":{
        "manifest_node_types" : [ "uberspark-uobj" ],
        "uberspark_min_version" : "5.1",
        "uberspark_max_version" : "any"
    },

    "uberspark-uobj" : {
        "namespace" : "uberspark/uobjs/generic/test/add",
        "platform" : "generic",
        "arch" : "generic",
        "cpu" : "generic",

        "sources" : {
            "h-files": [ ],
            "c-files": [ "add.c" ],
            "casm-files": [ ],
            "asm-files": [ ]
        },

        "publicmethods" : {
            "add" : [
                "uint32_t",
                "(uint32_t param1, uint32_t param2)",
                "2"
            ],

            "inc" : [
                "uint32_t",
                "(uint32_t param1)",
                "1"
            ],

        },

        "intrauobjcoll-callees" : {
        "uberspark/uobjs/generic/test/uobj1": ["pm_one", "pm_two", "pm_three"],
        "uberspark/uobjs/generic/test/uobj2": ["pm_one"]
        },

        "interuobjcoll-callees": {
            "uberspark/uobjcoll/test" : ["pm_one" ]
        },

        "legacy-callees": [
            "untrusted_func_1",
            "untrusted_func_2"
        ]

    }
}

uberspark-uobjcoll Manifest Node

The uberspark-uobjcoll node within the überSpark manifest is used to describe a üobject collection. The JSON declaration of the uberspark-uobjcoll node is as below:

uberspark-uobjcoll
Object Properties:
 
  • namespace (string) – <uobjcoll-namespace-path> as identified by überSpark (“<uobjcoll-namespace-path>”)
  • arch (string) – üobject collection target hardware platform
  • arch – üobject collection target CPU architecture (“x86_32”)
  • cpu (string) – üobject collection target CPU model (“generic”)
  • hpl (string) – üobject collection hardware privilege level (“generic”)
  • sentinels-intrauobjcoll (string list) – type of intra-üobject collection sentinel (“call”)
  • uobjs (uobjs) – list of üobjects within the üobject collection
  • publicmethods ({}) – comma delimited üobject collection public methods declarations of the format: “<uobj-namespace-path>” : { “<uobj-publicmethod-1-name>” : [ “<sentinel-type>”, …, “<sentinel-type>”], … , “<uobj-publicmethod-n-name>” : [ “<sentinel-type>”, …, “<sentinel-type>”]} (<sentinel-type>=”call”)
uobjs
Object Properties:
 
  • master (string) – <uobj-namespace-path> of the master üobject within the üobject collection (“”, “<uobj-namespace-path>”)
  • templars (string list) – comma delimited list of <uobj-namespace-path> for all the templar üobjects within the üobject collection (“”, “<uobj-namespace-path>”)

An example definition of the uberspark-uobjcoll node for a sample üobject collection called test, within uberspark.json follows:

{

    "uberspark-manifest":{
        "manifest_node_types" : [ "uberspark-uobjcoll" ],
        "uberspark_min_version" : "5.1",
        "uberspark_max_version" : "any"
    },

    "uberspark-uobjcoll":{
        "namespace" : "uberspark/uobjcoll/generic/test",
        "platform" : "generic",
        "arch" : "generic",
        "cpu" : "generic",
        "hpl" : "any",
        "sentinels-intrauobjcoll" : [ "call" ],

        "uobjs" : {
            "master" : "",
            "templars" : [
                "uberspark/uobjcoll/generic/test/main",
                "uberspark/uobjs/generic/test/add"
            ]
        },

        "publicmethods" : {
            "uberspark/uobjcoll/generic/test/main" : {
                "main" : [ "call" ]
            },

            "uberspark/uobjs/generic/test/add" : {
                "add" : [ "call" ]
            }
        }

    }
}