Development of Data Adapter Tools

See the project address https://github.com/goblin-pitcher/data-adapter

background

In the development process, it is often necessary to reduce the dependence on the back-end data structure to avoid large-scale modification of the front-end code caused by the change of the interface data structure. Therefore, an adapter tool is needed and unified management of conversion rules is required. When the interface data structure changes, it is only necessary to dynamically maintain the conversion rules.

demand analysis

First analyze IO, the methods that need to be implemented are as follows

 /**
* @params {Object} data 需要转换的数据
* @params rules 数据结构待定，转换规则
* @params {Object} options 转换配置
* @returns {Object} adaptedData 返回数据
*/
function adapter(data, rules, options) {}

Since the amount of returned data may be large, in order to avoid unnecessary overhead, it is better to modify the data in place, so the returned adaptedData satisfies adaptedData === data . If it is determined that the amount of data is not large and new data needs to be returned, it is best to pass in cloneDeep(data) when using it.

old version

The initial version is the product of the temporary requirements of the project. The conversion rules rules are a key for the matching path, and the value for the converted path or value. The specific code can refer to the address . Examples are as follows:

 /** 
* key为匹配路径, 如：
*  'e|a'代表匹配key为e或a，
*  'b.c'代表匹配路径['b', 'c']
*  '/^c/'代表正则匹配key值，匹配规则为/^c/
*  写法上是支持多种类型混用，如'e|a.b./^c/'，会匹配obj[e|a][b][/^c/]
* value可表示转换后的路径或值，规则如下：
*   当value为字符串待变转换后的路径
*   当value为方法时代表转换后的值，参数分别为：
*      data: 匹配路径的值
*      path: 匹配路径
*      obj: 原对象
*/
const rules= {
  "e|a": "b.a", // 将obj.e或obj.a的值放在obj.b.a下
  "b.c": "b.d", // obj.b.c的值放在obj.b.d下
  "/^c/": "b.f", // 将obj下以c开头的key放到obj.b.f下
  "b.ff": "b.g.f",
  e: (data, path, obj) => obj.a + obj.ca, // obj.e = obj.a + obj.ca
  "b.c": (data) => data ** 2, // obj.b.c = obj.b.c ** 2
};
const obj = {a:5,b:{g:{f:"xxx"},a:5,d:7,f:9},ca:8,cd:9}
adapter(obj, rules)

Since the old version is only a temporary solution, there are undoubtedly many problems.
Rule definition: for example b.c represents the path ['b', 'c'] , which will be ambiguous with b.c which is used as the key value. Extra escape character.

In terms of scalability: the old version only supports one configuration, namely retain - whether to retain the items before conversion. This part of the code is written in a hurry, and the coupling is too strong. Adding new configurations needs to modify many places, which is inconvenient to expand.

new version

First of all, in the definition of rules, in order to avoid many problems of the old version, the new version uses Map as a rule. The key is the matching path, and the value is the conversion rule.

How to use

For details, please refer to the test example

 npm i git+https://github.com/goblin-pitcher/data-adapter.git -S
--------
import adapter from 'data-adapter';
const obj = {
  a:5,
  b:{
    g:{
      f:"xxx"
    },
    a:5,
  }
}
const rules = new Map([
  ['a', 'transKey-a'],
  [['b', /a|g/, 'f'], (path, value)=>`transKey-${path[path.length - 1]}`]
])

// 转换后数据格式如下：
// {
//   'transKey-a':5,
//   b:{
//     g:{
//       'transKey-f':"xxx"
//     },
//     a:5,
//   }
// }
adapter(obj, rules)

The format of the adapter method is as follows:

 interface IOptions {
  retain?: boolean;
  transValue?: boolean;
  matchFullRules?: boolean;
  relativePath?: boolean,
  priority?: ('string' | 'regExp' | 'function')[];
}
// 当options为布尔类型时，代表配置{retain: options}
type RulesAndOptions = [rules: Rules, options: boolean | IOptions];
interface Adapter {
  (obj: Record<string, unknown>, ...args: RulesAndOptions): Record<string, unknown>;
  (obj: Record<string, unknown>, ...args: RulesAndOptions[]): Record<string, unknown>;
}
// adapter也可以接收多个转换规则，即adapter(data, [rules1, rules2, ....])

matching rules

Define the matching rule rules as a Map structure. Suppose the rules value is as follows:

 const testFunc = (path, value, matchPath, matchRule) => path[path.length - 1]==='f' && value>5;
const rules = new Map([
    [['b', /a|g/, testFunc], (path, value, matchPath, matchRule)=>`transKey-${path[path.length - 1]}`]
])

若以rules去转换data，其中一条rule的key是['b', /a|g/, testFunc] ，代表先data.b ， data.b.a data.b.g ， And look for the data.b.a and data.b.g that satisfies testFunc , if the item exists, convert its key to transKey-${key}

Configuration instructions

 interface IOptions {
  // 是否保留转换前的数据，默认为false
  retain?: boolean;
  // rule.value是否作为转换项的值，默认为false
  // 假设某条规则为new Map([['a', 'b']])：
  //   1. 若该项为true，代表data.a = 'b'
  //   2. 该项为false，代表data.b = data.a
  transValue?: boolean;
  // 是否匹配全路径，默认为true。
  // 比如某条规则为new Map([[['a', 'b'], 'xxx']]),假设data.a.b不存在：
  //   1. 当matchFullRules为true，则该条规则不生效
  //   2. 当matchFullRules为false，则会退而求其次寻找data.a，若data.a存在，则会转换data.a
  matchFullRules?: boolean;
  // 转换后的路径是否相对于转换前的路径，默认为false.
  // 比如某条规则为new Map([[['a', 'b'], 'xxx']])：
  //   1. 当relativePath为true，代表将data.a.b的值放到data.a.xxx下
  //   2. 当relativePath为false, 代表将data.a.b的值放到data.xxx下
  relativePath?: boolean,
  // 匹配优先级，默认为['string', 'regExp', 'function']
  // 比如某条规则为new Map([[['a', ['b', /^b/, testFunc]], 'xxx']])
  // 其中['b', /^b/, testFunc]代表以多种规则去匹配data.a下的所有项，priority代表匹配的优先级
  priority?: ('string' | 'regExp' | 'function')[];
}

Realize ideas

Because of the tight schedule, the old version was relatively poor at that time, the implementation was messy, and the scalability was poor. This function is implemented with a more reasonable data structure after refactoring.

Suppose the rule data is as follows:

 const testFunc = (path, value) => path[path.length-1].endsWith('b')
const rules = new Map([[[/a|e/, ['b', /^b/, testFunc], 'xxx'], 'transValue']]);
const data = {
    a: {
        b: {
            xxx: 7
        },
        ab: {abc: 4},
    },
    b: 5,
    e: {acb: {xxx: 6}}
}

It can be found that when there is an array ( ['b', /^b/, testFunc] ) in the rule item, there are various paths for the matching rule. For each matching path, it is possible to match the data of multiple paths. So define two tree structures :

• rule tree
• match data tree

The conversion process is shown in the following figure:

The data structure generated by rules is named rule tree

The rule tree and the data generated by the data structure are named match data tree

Various configurations can be easily completed by operating the rule tree and the matching data tree, such as matching priority options.priority , which can be achieved by modifying the order of children of each node in the rule tree; options.matchFullRules Configuration can be achieved by deciding whether to prune the matching data tree.