Refactoring practice of complex C-side projects

1. Background

1.1 Refactoring

Q: What is refactoring?

Refactoring is the improvement of the internal structure of software without changing its observable behavior. -- "Refactoring - Improving the Design of Existing Code"

Q: Why refactor?

Refactoring can improve understanding and reduce modification costs. -- "Refactoring - Improving the Design of Existing Code"

Q: When to refactor?

(1) When should not refactoring?

When there is no value, no meaning or low input-output ratio. Team resources are limited, and limited resources should be devoted to meaningful things as much as possible. Considering the input-output ratio from the team's point of view, don't touch the code that is already in a maintenance state, such as no demand or adjustment. If it is a novice, it will not bring benefits but may dig a hole. There may be many pits in the code.

(2) When should refactoring be done?

1. Project maintenance costs are high
2. Affect project tuning, such as performance optimization
3. When the code looks ugly and not elegant
4. When the existing design and implementation are not conducive to extending new functions
5. Repetitive work, when existing code doesn't help you easily add new features
6. Troubleshooting logic when patching bugs
7. code review allows others to review the code to check whether it is readable and understandable
8. Too much code is uncommented, and even I can't quickly sort out the code logic

1.2 How to refactor

(1) Preparation (basic skills)

It is recommended to read and re-read the classic book, Refactoring Bible "Refactoring - Improving the Design of Existing Code" . Since the first year of graduation, I have read it 4+ times in the past few years, and I have benefited a lot. Every time I review, I can gain something. The projects that I often toss and handle have not had any problems.

(2) Refactoring practice points

• Think clearly (there is a design as a whole, not necessarily documented but needs to be thought clearly).
• Collaborative planning (cooperation within the development team and integration of refactoring branches and other branches, advance application of external resources such as products, testing, operation and maintenance, etc.), overall planning.
• Layered and step-by-step, grasp the big and let the small from coarse to fine. Make good use of " batch ".
• Do one thing at a time.
• Don't reinvent the wheel.
• When you find something difficult, stop and think about whether the method is being used the wrong way, and how it should be. Keep monitoring and reviewing your way of thinking.
• Good internal and external communication, especially when the project is not developed and maintained by only one person. Pay attention to communicate with relevant parties (testing, operation and maintenance) in advance (plans, main time nodes, resources to be invested, matters that need their cooperation).

2. Refactoring practice on the C-side of the community

This reconstruction has a certain degree of complexity. In addition to the cost of technology migration and transformation, several warehouses involved are different technology selection (framework & upper-level components, etc.), rapid and agile project iteration, high demand for concurrency, and multi-person collaborative development. maintenance status.

2.1 Current Situation Analysis

Technology stack:

warehouse name	technology stack	The number of community C-side pages
repo A	React + umi3	The target warehouse does not need statistics
repo B	React + umi3	5
repo C	vue2 + vuex	27

project side

The three warehouses A/B/C are actively updated, and each warehouse involves the development of multiple lines of business and parallel maintenance. They are carried out according to the iterative rhythm of 2 weeks and a sprint, 1 week of development and 1 week of testing, occasionally interspersed with hotfixes.

The refactoring started after the release of the V1 main version. The codes involved in each repository are as follows:

1. repo A: A1 + A1. + A2 + A2.
2. repo B: B1 + B1. + B2 + B2.
3. repo C: C1 + C1. + C2 + C2.

.* means hotfix

2.2 Refactoring plan

The overall idea of the front-end side:

• Repo A is relatively new and is the main repository of the community, which concentrates most of the C-side pages as the target repository for the objective C-side code.
• Repo B to repo A: Repo B is very close to the technology stack of the target warehouse, involving 5 pages, migrated through human flesh, and pay attention to the migration of dependencies in the process.

• Repo C to repo A: Repo C is quite different from the target warehouse, and the languages are heterogeneous. The upper-level frameworks and component libraries are quite different, and there are many pages involved.

  • First, determine the valid pages, and exclude the dead code of the offline pages from the migration scope; the specific details will be mentioned later. Take out the front-end routing configuration in the warehouse to be migrated, know the total scope of the page, and check the recent Alibaba Cloud sls log. PV (two query methods for proofreading), exclude pages with no traffic.
  • Hierarchical and hierarchical reconstruction, focusing on large and small in the early stage, the framework syntax conversion (vue to react), which is time-consuming and labor-intensive and prone to problems, should be implemented using scripting tools to maintain the overall structure and reference relationship at the file level and in each class. conversion.
  • The detailed syntax is batched by custom scripts (for example, the key of the class used in vue and the value in the form of a string are converted into the className and the value in the form of a variable in react).
  • In order to ensure efficient self-testing after migration, it is necessary to keep the corresponding *.vue files, treat them as doc files, and delete them after the entire migration is completed to improve the efficiency of migration and testing. Note that retrofitting lint rules ignores detection of such files.
  • During the process, the dependent files are moved in together, and there is "name space isolation". Pay attention to maintaining the relative relationship of the overall directory structure, do the overall migration, and do not pollute the existing files in the target warehouse to prevent files with the same name from being overwritten.
• After migrating each repository code to repo A through the above three steps, synchronize the latest updates in the three repositories. In the process from repo C to repo A (the branch cut from V1), repo C is still updating the code, and repo A also needs to merge the V1. , V2, V2. code in repo C (the same is true for repo B) . Since the code is all in different repositories, it needs to be merged manually. Tips : You can combine multiple commits of V1. , V2, and V2. into one commit in repo C, and aggregate all the changes into one place for batch update.
• The research and application of the SSR scheme in repo A are also in parallel. The newly migrated pages in the refactoring need to be integrated with SSR.

2.3 Refactoring and Integration Practices

2.3.1 Warehouse B page sorting and moving in

This part of the migration is carried out in an isomorphic language and involves a small number of pages, mainly through human migration.

2.3.2 Warehouse C page sorting and moving in

• Online traffic query to exclude useless pages

  • Routing declarations in the three code repositories determine the total scope

  • Determine the PVs in the past 3 months, 2 months, and 1 month according to Alibaba Cloud logs, and remove pages without PVs from the page pool to be migrated.

    • Note 1: Alibaba Cloud SLS logs are based on reported data. Data may be lost during reporting and statistics. Therefore, two query entries are combined to determine and troubleshoot.
    • Note 2: For pages with 1-2 PVs, it may be generated when the team conducts research in the early stage of development. After confirming the visitors, the "test" pages that generate PVs are discharged.

• Determine the final reconstruction range (13 of 27 filtered). Pages without user PV in step 2 are removed from the total range obtained in step 1.

• Heterogeneous language conversion and processing

  • Vue2 in warehouse C is converted to react in warehouse A

    • Tool conversion

Vue-to-react is mainly used here. However, this tool has many constraints and limitations. About half of the code has been successfully converted. If the conversion fails, you need to write your own script to implement it. I originally wanted to encapsulate and transform the source code of the library. After reading its implementation, I found that the cost of customization is higher than the cost of writing scripts myself, so I abandoned it (my experience with Vue is less than a month), and the time is too tight to study it carefully. . Tips : Avoid repeating the creation of wheels. When performing tedious and repetitive actions, you can consider embracing the internal wheels, community and open source of the team. If not, you can make one yourself.

• Script conversion

• convert

  • Project directory structure design and file mapping process

 // step1：保持整体目录结构的相对性不变

.

├── apis

│   ├── community.ts

│   ├── h5community

│   ├── ...

├── components

├── pages

│   ├── h5community

│   │   ├── App

│   │   ├── api

│   │   ├── asset

│   │   ├── components

│   │   ├── config

│   │   ├── filter

│   │   ├── live.js

│   │   ├── main.js

│   │   ├── mixins.js

│   │   ├── router

│   │   ├── style

│   │   ├── utils

│   │   └── views

│   ├── community

├── utils

└── ...




// step2: foo.vue文件转为 foo/ 目录，模板分别映射为jsx及less文件

.

├── apis

│   ├── community.ts

│   ├── h5community

│   └── ...

├── components

│   ├── h5community

│   └── ...

├── config

│   ├── h5community.js

│   └── ...

├── pages

│   ├── community

│   └── h5community

│       ├── column  // 原 column.vue 转为目录，分拆成index.tsx及index.scss

│       │   ├── index.local_js // index.local_js作为注释保留，用于测试回归的参考

│       │   ├── index.scss

│       │   └── index.tsx // 首行自动插入对 index.scss 的引用

│       └── ...

└── utils

    ├── h5community

    └── ...

• Step-by-Step Conversion 1: File Level

For pages that fail to be processed by vue-to-react, generate page template files through scripts.

 // 转换前文件为 foo.vue


// 转换后：
.
└── foo
    ├── index.jsx
    ├── index.local_js
    └── index.scss

The content structure of the file generated by the custom script conversion is as follows:

• Step-by-step transformation 2: Syntax level - html lang

There are many templates of lang="pug" class in the Vue file conversion process, which are converted into "jsx-like" templates through the tool https://pughtml.com/ (as long as it is tasteless, you should first think of the tool, if you can't find it , might as well try different keywords in Google instead of manually).

 // 转换前 foor.vue 中

<template lang="pug">

    article.modal-wrap(@touchmove.stop.prevent @click.stop='close')

        section.modal

            p.more 更多精彩内容, 就在得物App

            p.slogan 有毒的运动 x 潮流 x 好物

            .enter-btn(@click.stop='enter') 进入得物App

            aside.close(@click.stop='close')

</template>







// 转换后 foo/index.jsx  中

<article class="modal-wrap" @touchmove.stop.prevent="@touchmove.stop.prevent" @click.stop="close">

    <section class="modal">

        <p class="more">更多精彩内容, 就在得物App</p>

        <p class="slogan">有毒的运动 x 潮流 x 好物</p>

        <div class="enter-btn" @click.stop="enter">进入得物App</div>

        <aside class="close" @click.stop="close"></aside>

    </section>

</article>

• Step-by-step transformation 3: Syntax level - className etc.

The files generated by the above script are file-level conversions, and the syntax differences need to be resolved by the script. Such as class replacement and parsing. The rule parsing regularity of the html attribute here is relatively cumbersome. When implementing it, you will think about where it will be. It is natural to think that there must be this regularity in the source code of vue (the framework needs to be parsed for native mapping). Just modify it, and then do some customization (the template code in the business code, such as import style, is automatically generated by script and inserted on demand).

 // foo.vue 文件中的写法 

<div class="var1">demo1</div>

<div class="var1 var2">demo1</div>







// foo/index.jsx （react中）的写法

import style from './index.scss'

import classNames from 'classnames'

...

<div className={style["var1"]}>demo1</div>

<div className={classNames(style["var1"], style["var2"])}>demo1</div>

• Page-by-page debugging and proofreading

• Differences between warehouse technology selection

  • Routing rules and customization of umi
  • Third-party component libraries such as Swiper, postcss-px-to-viewport, etc., the vue version is somewhat different from the react version, the documentation is incomplete, and the source code and community are embraced. Among them, postcss-px-to-viewport uses different viewportWidth settings in different warehouses. During the conversion process, it is realized by processing different path ranges for different plug-in instances.
• Basic skills: Sensitivity (this is related to experience). What is library positioning? How is the maturity? What should and should not support? If you design it yourself, how will it be designed (sometimes even if the documentation is incomplete, you can deduce a lot of content without looking at the source code)? Where can I go to find a solution? How to find it?

• Migrate home page configuration

  • During the process, the path scope of the home page is narrowed, the access path in repo A is hidden, and only the path to be migrated is revealed, which improves the search efficiency
• Migration process records (test data and paths, etc., to facilitate cross-testing and QA regression)
• Coverage self-test. In the case of multiple business logics in a page, sufficient self-testing of each path is required in the future
• Design and change path of directory and file structure during migration (important)

2.3.3 Integrate repo A, repo B, and repo C to refactor branch code

• Migrate pages in repo B to repo A, such as branching with chore-repoB
• Migrate pages in repo C to repo A, such as branching with chore-repoC
• Merge the repo A master branch with chore-repoB and chore-repoC and resolve the conflict. The merged branch is recorded as chore-repoA-repoB-repoC. At this time, this branch only has the code of V1, and the iterative functions of the current version of each warehouse and above version of the hotfix has not been merged into this branch.

2.3.4 Integrate iterative branch code in repo A, repo B, and repo C

On the afternoon of the day before the main version, the iterative functions in various warehouses were basically stable, and the bugs had converged. At this point, the local development branches feat-foo, feat-bar, etc. of each warehouse can be aggregated into a pre-release-temp branch (which already contains the hotfix on the master), that is, the pre-release-temp branch is V1. , V2 summary, synthesize the incremental commits of the branch into one commit to obtain the file changes affected by V1. , V2. Artificially sync these changes to repo A chore-repoA-repoB-repoC branch.

2.3.5 Integrate three warehouse business codes and SSR codes

After the community C-side SSR transformation plan was determined, a new A-SSR warehouse was opened. Use the framework content of the SSR POC to initialize the A-SSR repository, and then migrate the code in chore-repoA-repoB-repoC in repo A to this repository. Problems encountered: Some modules in the original repo A have been SSR converted in the POC, and the new code has been migrated to the repository. Note that the file coverage code is lost. Use cp, then git diff and artificially check to change the source files before submitting. .

It will be nearly 1 day before the version date + the bugfixes generated by each repository will be synchronized to the A-SSR repository to ensure that the code is not lost .

3. External collaboration for project promotion

3.1 Testing

Large-scale refactoring needs to ensure sufficient testing. Considering the occupied testing resources, communicate resource requirements with the testing leader as early as possible. In addition, before moving the test, try to fully self-test inside the front-end.

3.2 Operation and maintenance

Plan the page redirection scheme in advance (redirect the final cross-warehouse/application migration page), pay attention to the impact of changes on the operation and maintenance side, once the changes are made, the relevant test environment will not be available (QA regression requires time, which affects the usage of the corresponding page on that environment if redirection is enabled).

3.3 Problems encountered

When planning and launching the refactoring, no one on the team was familiar enough with all three C-side repositories involved. When migrating to the second page, when it was found that some pages were dead code with no online traffic, they re-communicated with the client and operation and maintenance students, and finally narrowed the migration scope by querying Alibaba Cloud sls logs, reducing the workload by nearly half. Various technical problems encountered in the process still need to be accumulated more often.

4. Summary

The reconstruction of complex projects has certain requirements on the basis, experience, specifications and coordination of all parties in research and development. At the beginning, you can read "Refactoring" several times to lay the foundation, and gradually start the refactoring of code modules, simple projects, complex projects, and complex cross-team projects, and accumulate experience. Make a plan in advance (the overall plan on the technical side, the advance prediction of difficult diseases on the technical side, the overall promotion plan, the participation of relevant parties, etc.), think comprehensively and carefully in the process, and continue to review and adjust, and do a good job of summarizing and precipitation after the process.

Designing in advance, regular Code Review, and continuous refactoring during and after the process can make the project code more maintainable. The team maintains the habit of refactoring while continuously accumulating refactoring experience, which can improve the overall performance of the project. Health and maintainability. Refactoring can see improvement is the key, grow in refactoring, benefit from refactoring, and benefit from refactoring.

Related Links:

• https://pughtml.com/

* Text / SHI FEI

Pay attention to Dewu Technology and be the most fashionable technical person!