Declarative Mapper is a JSON data transformation and object mapping library for JavaScript/TypeScript. Define declarative mapping templates to convert documents quickly, with schema helpers and safe VM-based execution.
On several projects, I needed a library that could convert one JSON format to another (for example, an invoice from one system into another). It had to support declarative mapping instructions so users could configure mappings from a UI. It also had to be flexible enough for complex requirements, secure against JS injection, and fast enough to process streams with millions of records.
That is where Declarative Mapper came in:
- Declarative - declarative mapping instructions allow configuration from a UI. In simple scenarios, no technical knowledge is needed.
- Flexible - runs JavaScript under the hood to support complex instructions.
- Secure - restricts access to the outside environment by executing mappings in a separate V8 Virtual Machine context.
- Fast - mapping instructions are compiled once up front, allowing processing at ~200k objects/sec on Apple M1 Pro.
- Typed - written in TypeScript
- Lightweight - no dependencies
import { createMapper } from 'declarative-mapper';
// Source records from system A
const sourceOrders = [
{
orderId: 'SO-1001',
customerName: 'Acme Ltd',
lineItems: [{ sku: 'A-1', qty: 2, unitPrice: 10 }]
},
{
orderId: 'SO-1002',
customerName: 'Globex',
lineItems: [{ sku: 'B-9', qty: 1, unitPrice: 25 }]
}
];
// Compile once
const mapper = createMapper({
id: 'orderId',
customer: 'customerName',
items: {
forEach: 'lineItems',
map: {
code: 'sku',
quantity: 'qty',
amount: 'qty * unitPrice'
}
},
totalAmount: 'lineItems.reduce((sum, i) => sum + (i.qty * i.unitPrice), 0)'
});
// Use in a loop; 200k+ objects/sec
const results = sourceOrders.map(mapper); - Node.js: 16+
- Browser: best effort support; requires a
vmpolyfill
Similar mappings can be achieved with plain JavaScript, but this library is designed for a different case: user-controlled mapping templates executed on the server.
Mappings stay simple for non-technical users, while technical users can still use JavaScript expressions.
Instead of eval, expressions run in an isolated VM context with built-ins, mapping input, and explicit extensions only, which reduces JS injection risk.
In mapping JSON, the left side is a key in the resulting object. The right side is either a string with a valid JS expression or an object with mapping instructions.
{
"key": "100", // numeric value, produces `"key": 100`
"key": "true", // boolean value, produces `"key": true`
"key": "'text'", // text value, produces `"key": "text"` (notice inner quotation marks)
"key": "foo", // access to an input variable `foo`
"key": "Number(foo)", // access to an input variable `foo` converted to a number, produces `"key": 100`
"key": "arr.map(e => ...)", // more complex JS expression that produces an array
"key": { // object mapping, produces `{ foo: 'bar' }`
"foo": "'bar'"
},
"key": { // same as above, but more verbose
"map": { /*...*/ }
},
"key": { // array mapped from input
"forEach": "nested.array.filter(e => !!e)",
"map": { /*...*/ } // $record, $index, $collection are available in "map"
},
"key": { // tuple array
"0": { /*...*/ },
"1": { /*...*/ },
"5": { /*...*/ }
},
"key": { // object mapping from a different context
"from": "some.nested.field",
"map": { /*...*/ }
},
"${prefix}_${id}": "value", // dynamic output key (template interpolation)
}| Variable | Description | Available in |
|---|---|---|
$input |
Entire source document passed to the mapper. | All mapping contexts |
$record |
Current element in a forEach iteration. |
forEach → map |
$index |
Current element index in a forEach iteration. |
forEach → map |
$collection |
Entire source array selected by forEach. |
forEach → map |
Mapping of an object with inner properties:
"key": {
"foo": "-1"
}or
"key": {
"map": {
"foo": "-1"
}
} Both examples above produce the same result (the second one is more verbose, but keeps a consistent format with array mappings):
"key": {
"foo": -1
} Assume we have input with an array of objects and need to produce one output object per element. In such cases, the "forEach": "", "map": {} construction can be used:
{
"inputArray": [{
"arrayInnerProp": "value1"
}, {
"arrayInnerProp": "value2"
}]
} "key": {
"forEach": "inputArray",
"map": {
"foo": "arrayInnerProperty"
}
}Result:
"key": [{
"foo": "value1"
}, {
"foo": "value2"
}]In this mapping, the execution context shifts to each input object, so inner properties can be referenced directly as arrayInnerProperty instead of inputArray[index].arrayInnerProperty.
Inside forEach mappings, $record, $index, $collection, and $input are also available.
More on that in Context Switching.
If the array should contain plain values instead of objects, use "*" on the left side instead of a key name:
"key": {
"forEach": "inputArray",
"map": {
"*": "arrayInnerProperty"
}
}Produces:
"key": [
"value1",
"value2"
]Arrays with simple values can be mapped the same way, and the current iterated element is available as $record:
{
"inputValues": [1, 2, 3]
}{
"forEach": "inputValues",
"map": {
"*": "$record * 2"
}
}Result:
[2, 4, 6]If the array should have a predefined set of elements, each element can be mapped by index:
"key": {
"0": {
"foo": "\"text1\""
},
"2": "1000"
}Result:
"key": [
{
"foo": "text1"
},
null,
1000
]When arrays are mapped with the "forEach": "", "map": {} statement, the execution context automatically switches to the objects selected by forEach (see above). A similar technique is useful when many properties need to be mapped from an object outside the current context. In that case, use the "from": "", "map": {} statement.
Down in the source tree:
"key": {
"from": "field.innerArray[0].innerObject",
"map": {
"foo": "nestedProperty"
}
}Or up in the source document:
"key": {
"from": "$input.rootLevelProperty",
"map": {
"foo": "nestedProperty"
}
}Inside from mappings, you can still reference root-level fields through $input.
Runtime variables:
$record- current element of the array being iterated withforEach$index- index of the current array element$collection- entire collection of the elements being iterated$input- entire document passed as mapping input
Combined example (forEach + root reference):
{
"forEach": "LINE_ITEMS",
"map": {
"lineNo": "$index + 1",
"sourceId": "$input.id",
"raw": "$record"
}
}You can build output property names dynamically with template-based keys on the left side.
This works in regular mappings, forEach mappings, and from mappings.
{
"${prefix}_${id}": "value"
}For input:
{
"prefix": "item",
"id": 7,
"value": "abc"
}Output:
{
"item_7": "abc"
}To keep ${...} as a literal key (without interpolation), escape it with a leading backslash:
{
"\\${prefix}_${id}": "value"
}Use extensions to pass helper functions and lookup data into mapping expressions:
const mapper = createMapper({
code: 'catalog[itemId]',
normalizedName: 'normalize(name)'
}, {
extensions: {
catalog: { A1: 'SKU-001' },
normalize: (s: string) => s.trim().toUpperCase()
}
});Extension keys are available as globals in expressions.
If an extension key conflicts with an input field name, mapper throws an error.
// Some kind of a document we expect on input
const input = {
LINE_ITEMS: [
{ UPC: '123', QTY: 1, PRICE: 3.4 },
{ UPC: '456', QTY: 2, PRICE: 5.7 }
],
ALLOWANCES: [
{ ITEM_UPC: '123', AMOUNT: 1.5 }
]
};
// Additional information we want to pass to the mapping environment
const itemCatalog = [
{ upc: '123', vendorCode: 'X-123' },
{ upc: '456', vendorCode: 'X-456' }
];
// Some format we need
const desiredOutput = {
title: 'Invoice 1',
items: [
{
code: 'X-123',
qty: 1,
price: 3.4,
amount: 3.4,
allowances: [1.5]
},
{
code: 'X-456',
qty: 2,
price: 5.7,
amount: 11.4,
allowances: []
}
],
total: 14.8
};
// Declarative instructions on how to convert the input format
// to the desired format
const mapping = {
// mapping to a constant
title: '"Invoice 1"',
// array mapping from another array
items: {
forEach: 'LINE_ITEMS',
map: {
// data lookup from an additional source passed to `extensions`
code: 'itemCatalog.find(e => e.upc === UPC).vendorCode',
// fields mapping in a context of `LINE_ITEMS` elements
qty: 'QTY',
price: 'PRICE',
amount: 'QTY * PRICE',
// data mapping from a source different from the current mapping context
// (ALLOWANCES are placed next to LINE_ITEMS in the input)
allowances: {
forEach: 'ALLOWANCES.filter(a => a.ITEM_UPC === UPC)',
map: {
'*': 'AMOUNT'
}
}
}
},
// property mapping from an array,
// with a custom reducer function defined in `extensions`
total: '$sum(LINE_ITEMS, i => i.QTY * i.PRICE)'
};
// Pre-compiled function that can be executed any number of times
const mapper = createMapper(mapping, {
extensions: {
itemCatalog,
$sum: (arr, cb) => arr.reduce((t, el) => t + cb(el), 0)
}
});
const result = mapper(input);
expect(result).to.eql(desiredOutput);