The business practice of Youdao interactive content engine Ceramics

Preface

Ceramics is an HTML5 interactive content engine, which mainly provides Web interactive topic development solutions for Youdao interactive teaching scenarios to help developers quickly produce high-quality topic content.

This article will start from the business scenario, introduce the technical implementation of Ceramics, and describe how it efficiently empowers the production of high-quality interactive content, hoping to provide some references for front-end developers who are interested in Web interactive technology.

1. Background

1. Interactive content

Interactive content is an important part of Youdao's online teaching, with a wide range of application scenarios such as:

• In the live classroom , the interactive content is pushed to the student end through the teacher's control, and the students can receive the interactive content when watching the live broadcast or after-class replay, and can submit the answer to the teacher after answering;
• In AI teaching video , interactive content is interspersed, and different follow-up video clips are played according to the user's answer results;
• In the App and the small program , after class, users can consolidate their knowledge points through review, free practice, and wrong question books. The tutor can view the user's answer in the background for special guidance;
• Intersperse interactive questions on the external cooperation, marketing activities, user pre-school test and other pages to pull new traffic.
  Interactive topics stimulate interest in learning and attract attention through interesting interactive processes and timely and reasonable feedback, allowing users to acquire knowledge more efficiently and consolidate learning in interactive and exploratory fun classrooms.

Figure 1: Application scenarios of interactive content

2. Subject template

An interactive topic is composed of topic templates and topic data: R&D personnel develop topic templates that include interactive logic, animation effects, and judgment of right and wrong according to the needs of teaching and research. Teaching and research configure topic data according to the data structure of the topic template.

At present, the team has developed nearly 300 question templates (hereinafter referred to as question boards), covering fill in the blanks, select, drag and drop, draw, connect and other question types. Through a good interface design and data structure definition, a topic template can be applied to a variety of teaching modes, and the topic templates can also be combined with each other to form a more interactive topic template.

Figure 2: The relationship between interactive topics, topic templates, and interactive engines

Taking the two types of click-and-remove questions in Figure 3 as an example, has differences in judgment logic, scene style, and animation effects , but the interactive similarity of the questions is relatively high. Based on this, the two types of questions are developed with a question template to retain a high degree of configuration. The data structure defines fields such as background pictures, option pictures, dynamic effects, judgment mode, question stem text, etc., and you can pass a question template Support multiple questions with different gameplay.

Figure 3: One question template supports multiple questions

3. Interactive Engine

Interactive engine is the basis of interactive research and development, a good interactive engine can greatly reduce research and development costs. As can be seen from Figure 2 above, the interactive content engine provides capabilities such as rendering, user interaction, animation, sound, and resource management. Based on it, developers can develop question boards that meet business scenarios. The question boards are based on different configuration data. Render out customized topic content.

In order to cope with the rapid development of the business and ensure the efficient and stable production of high-quality interactive content, the front-end team of the children's project team has self-developed a high-performance, well- , and easy-to-develop interactive content engine: 160e5823716aed Ceramics . Ceramics is an HTML5 interactive engine developed based on TypeScript+React, including user interaction, sound, animation, rendering, resource management and other functions. Web developers can use the API and components provided by it to quickly develop it with low learning costs. Interactive question board for iOS, Android, Web, and H5 multi-terminal adaptation.

Since Ceramics was put into use, it has successively provided content production services for the multidisciplinary and multiple business lines of the children's group. The following will introduce the technical implementation of Ceramics from the aspects of architecture design, user interaction, animation effects, component design, etc., hoping to provide some references for front-end developers who are interested in interactive technology.

2. Technology and Realization

1. Infrastructure

First, let's take a look at the overall architecture of Ceramics.

Figure 4: Ceramics architecture

In simple terms, the interactive engine is a tool for rendering topic data to the interface, which is composed of a data layer, a view layer, and a business logic layer. The data drives the view, and the rendering engine renders the data to the canvas according to the logic.

The data is stored in the configuration file in the form of JSON

• UI layer configuration : such as page layout, number of elements, size, positioning, etc.
• script configuration : such as the timing of element appearance, the sequence of animation playback, user interaction behavior, feedback, etc.
• resource configuration : such as background images, sound effects, dynamic resource addresses, etc.
• topic data configuration : topic specific information, such as copywriting, topic stem, pictures, answers, reading voice, etc.

The main work of web developers is in the view layer and business logic layer, that is, how to visualize the configuration file, including the core logic of the question board, the realization of specific interactions, and the calculation of the judgment. We usually implement complex animation effects and drawing in the Canvas area, use DOM to implement layout, and use TSX+hooks to write logic code. In order to reduce the workload of this step, the engine itself provides:

• Animation capabilities : Including the display and control of CSS animation, JS animation, skeletal animation, and AE animation, and encapsulate common animation effects into an animation library.
• Audio playback capability : Support the playback of mp3 audio files and the playback of speech synthesis tts files.
• rendering capabilities : 2D and 3D rendering based on HTML, Canvas, and WebGL.
• user interaction : encapsulates user interaction events such as dragging, pressing, clicking, input, handwriting, cutting, and 3D interaction.
• adaptation and compatibility : Well-tested mobile terminal adaptation and compatibility, supporting iOS 9 (Safari 9), Android23 (Chrome 44), Web, WeChat applet and other platforms.
• performance optimization : The engine is optimized through lazy loading, resource packaging optimization and other means, and the reliability of the engine is guaranteed through Sentry monitoring and 3D animation frame rate detection.

Based on the above basic capabilities, we encapsulated some commonly used atomic components and functional plug-ins, such as handwriting pads, calculators, abacus, etc., as well as gamification commonly used barriers, countdowns, guide animations, etc.; also encapsulated general question board components to achieve The logic and interaction of basic question types such as dragging, selecting, filling in the blanks, and connecting. Atomic components can be combined and packaged into functional plug-ins, and common question board components and functional plug-ins can be combined to form business question board components.

The above capabilities and components, as well as detailed access documents and development documents, lower the development threshold of interactive question boards. Web developers do not need to learn 2D, 3D, graphics, and game development knowledge in advance, and they can quickly start developing those that meet business needs. Interactive question board.

2. Interaction ability

Interaction capabilities are the basis for the interaction between users and the screen. Common user interactions include dragging, pressing, clicking, input, handwriting, cutting, interaction in 3D scenes, etc. The interactive question board can be used on both the Web and mobile terminals. Also consider the interaction of mouse and finger touch.

Ceramics uses the popular ECS (entity-component-system) design pattern in game development as the underlying architecture in the design of interactive capabilities.

One of the core concepts of ECS is that composition is better than inheritance. The immutable part uses inheritance to facilitate reuse, and the changeable part uses composition to facilitate expansion. This design pattern can ensure the high scalability of the engine.

In interactive content, each element (small animals, grass, stones, balloons, etc.) is an entity. Each entity can be bound to one or more components, and each component describes different capabilities (moving, dragging, etc.) , Click, etc.), to assign one or more abilities to the entity through combination, or dynamically add or delete abilities.

Take the simplest button as an example. It is an entity without any capabilities and is represented by a <div/> element. Buttons can have push buttons and switch buttons, which are used in different scenarios. We have customized the usePress hook and useToggle hook, uniformly handle the mouse and touch interactions on the web and mobile terminals, smooth out the differences between browsers, and expose simple APIs; the useButton component is encapsulated based on usePress, indicating that the button can be pressed Capability; encapsulates the useToggleButton component based on useToggle and useButton, which represents the basic capabilities of the switch button. Finally, bind useButton or useToggle to the button component to become the Button component and the ToggleButton component, which gives the button corresponding capabilities.

Figure 5: Button component interaction capability design

For example, the drag interaction, as shown in Figure 6, is simply the behavior of dragging an object to a specified hot zone. True or false judgment is to compare the object in the hot zone with the object in the answer setting at the specified time.

Figure 6: Drag and drop objects and hot

Draggable objects and non-dragable objects are essentially entities that are equally incapable. Encapsulate the drag ability in a separate component, handle the movement behavior of mouse and touch, and the collision detection between dragged objects and hot spots. The draggable object is given the draggable ability, the target hot zone is given the droppable ability, and the subsequent logic processing is performed through the interface exposed by the drag-and-drop component to realize the dragging behavior.

In addition to the realization of interactive capabilities, the design ideas of ECS are reflected in many modules of Ceramics, such as expressing capabilities through custom hooks, which can achieve a greater degree of logic reuse. There are many custom hooks in Ceramics, such as useQuestion hooks, which integrate the question-making process, including verifying answers, feedback based on the verification results, and accumulating the number of errors; useViewport hooks, which encapsulate the adaptability of the mobile terminal; useAnime hooks, Encapsulates the animation operation capabilities, such as play, pause, replay, etc. Introducing the same hooks in different question boards can unify the code style, improve development efficiency, and new developers can understand it better and make it easier to use.

Figure 7: useQuestion hooks

3. Animation effects

Animation is an important part of interactive content. Realistic and interesting animation effects can not only assist in displaying teaching content, but also help enhance user immersion, attract attention, stimulate interest in learning, and better retain users.

Common web animation solutions mainly include sequence frames, CSS animation, JS animation, Canvas, WebGL, etc. Ceramics supports multiple animation forms by introducing commonly used animation libraries, and performs secondary packaging for business scenarios and compatibility. For some simple animations, R&D only needs a simple API call to achieve the effect. Some front-ends implement costly and complex motion effects. Designers use AE, Spine and other tools to produce, and develop and introduce corresponding components in Ceramics to achieve animation rendering and control.

3.1 Lottie

Lottie is a motion solution developed by Airbnb that can be applied to multiple platforms such as Web, Android, and iOS. After the animator uses AE (After Effects) to create an animation, the AE Bodymovin plug-in can export the json file that renders the animation on the mobile terminal and the web terminal. The front end can parse and render the json file containing animation information through the Lottie library. Lottie animation is simple to use, efficient and high-performance, strong in effect restoration, and good cross-platform support. It is well presented on iOS, Android, Web, and small terminals.

Figure 8: Lottie animation

3.2 Spine

Spine is a commonly used 2D skeleton animation production tool in game development. The animation produced by Spine can be exported as a json + atlas + png file. Spine Runtime is a library officially provided by Spine that can parse animation files exported by spine. HTML Canvas runtime provides the ability to render Spine animations on the Web.

Spine animation can be applied to different engines and languages. Therefore, in business lines that require cross-terminal collaboration, designers can use a set of spine animations to adapt to different applications to improve work efficiency and avoid duplication of work.

Figure 9:

3.3 Animation library based on Anime.js

Anime.js is a lightweight Javascript animation library that can animate CSS properties, SVG, DOM and JavaScript objects. Anime.js's built-in interlacing system makes complex animations and overlapping animations simple.

Based on Anime.js, Ceramics has developed a motion effect library that is more in line with business needs, and encapsulates the general motion effects of the question board, such as:

1. Emphasis (zoom, change style, jitter, blink)
2. Element entry and exit animation (fading in, fading out, flashing, falling)
3. Element entry and exit animation (fading in, fading out, flashing, falling)

The animation library is also an independently released npm package, which can serve multiple projects and improve development efficiency. The animation components in @ceramics-ae are directly introduced in the project, and the effects can be achieved by adding animation names and parameter configurations DOM Use timeline single-step animations in series, supporting single-play, loop-play, timeline and other functions.

import { Teeter } from "@ceramics-ae/core"
Teeter.play({ el }) //抖动
StyleChange.play({el,config: {style: { color: "#F5636C",background: `center/100% no-repeat url(${OptionStyle[type].bg_wrong})`}}}) //更改css样式

3.4 WEB 3D

To achieve 3D effects on the Web, the most popular technology currently is WebGL. WebGL is based on OpenGL, which is a cross-platform 2D and 3D drawing standard. WebGL combines OpenGL and JS to display 3D scenes and models on the Web.

WebGL is powerful and provides a large number of basic drawing APIs, but the threshold is high, and the code complexity is high. Developers need to have prior graphics knowledge and mathematical knowledge.

Three.js is a WebGL open source engine that encapsulates the underlying rendering details and complex data structure of WebGL. Developers can configure lights, vertex shaders, fragment shaders, and automatically generate matrices through simple API calls. View angle control, rays, etc., make 3D models quickly. We have made 3D effects such as Rubik's Cube and 3D graphics based on Three.js. The development is simple, the effect is high, and the mobile terminal can also maintain good performance and compatibility.

Figure 10: web 3D

A set of high-quality animations is often the result of the combined efforts of vision, animation artists, and front-end development. Developers can choose different animation schemes according to different application scenarios, material dependencies, development costs, performance and compatibility. While reducing the cost of collaboration between designers and R&D, the animation effect of the new design improves team work efficiency.

4. Question board component design

Ceramics, as an interactive content engine, one of the goals is to efficiently produce topic templates for multiple business parties. We hope that each topic template can serve more business parties, and can receive more quickly when new business parties have needs. Into.

Based on this, we encapsulate each question board into a question board component, develop it independently, and publish it as an npm package; the question board is decoupled from the business, and the question board is decoupled from the question board. The upper-level business party is responsible for controlling multiple question boards. Logic such as the combination and switching of the question board; the question board component can also be used as a sub-component to be introduced by other question boards for secondary packaging, which improves the reusability of the question board to a greater extent.

Figure 11: The calling relationship between the question board component and the business party

Usually when the upper-level business party uses the question board, it needs to collect the internal state of the question board to realize the external business logic. For example: when the question board is loaded successfully, the transition animation and background music are removed, the user answers the wrong question book when it is wrong, and it switches when the answer is correct. For the next question, the question board also needs to update the status according to external data. For example, if the user enters the question in different game modes, the question board should switch the corresponding interface and logic.

For the above business scenarios, the question board component defines a set of common interfaces:

Figure 12: Question board component interface

The upper-level business party installs the npm package, introduces the question board as needed, and passes in the props according to the interface definition to complete the access.

import { CeramicsProvider, DragCetegoryQuestion } from "@ceramics-math/core"
<CeramicsProvider>
  <DragCetegoryQuestion //题目模板组件
    questionConfig = {}  //题目配置数据
    deviceConfig = {}/> //设备配置数据
</CeramicsProvider>

The question board components are independent of each other, but have similar logic and component dependencies. In order to better reuse code and manage project dependencies, we have introduced the Lerna framework for multi-package management, combining multiple question board components as sub-projects into a parent project, and repetitive code logic is also extracted into another parent project. An independent sub-project.

Figure 13: The structure of multi-package management

benefits brought by multi-package management mainly include:

• All sub-projects can share the same package and test configuration;
• Within the sub-project, other sub-projects can be introduced in the same way as other dependent packages. Lerna will automatically identify and guide the internal projects, and synchronize updates in real time;
• It is more convenient to handle the dependencies of dependent packages. Install the dependent packages of all sub-projects through the Lerna bootstrap command. When installing dependencies, increase the dependencies, that is, upgrade all project npm dependent files to the root directory to avoid installing the same dependent packages in different projects repeatedly.

Through Lerna commands, you can easily execute the build, test, and component release processes of all components, update and release the corresponding dependent version numbers in the components, and tag and generate changelogs after the components are released.

In addition, because different question boards have more similarities in package release configuration, interface definitions, mobile terminal adaptation rules, component dependencies, etc., in order to facilitate new colleagues to get started, avoid interface omissions, and unify code style and data structure, we also Defined the initialization template, and introduced the automated construction tool Plop to assist in the rapid creation of question board components. Through interactive command line input, you can dynamically generate question board components in the specified directory, including initialization code and configuration files.

Figure 14: Interactive command line

Figure 15: Initialization template and generated file
As described in this chapter, clear interface definitions, convenient and quick access methods, multi-package management and the introduction of automated construction enable upper-level business parties to quickly access the question board component library, and the development, iteration and maintenance of the question board components are also much easier.

5. Atomic Components and Plugins

In order to further improve development efficiency, we have encapsulated many atomic components and functional plug-ins according to common business scenarios. Developers can introduce them through simple component introduction and parameter transfer.

Figure 16: Atomic components and function plug-ins

We understand the smallest elements in the interface as atomic components, such as buttons, titles, display boards, options, progress bars, etc., according to different configuration styles and interactions of props.

Figure 17: Atomic component

Functional plug-ins are composed of multiple atomic components, coupled with some business logic, commonly used plug-ins such as handwriting pads, calculators, abacus, etc., as well as gamification commonly used to pass through, countdown, and guide animation.

Figure 18: Function plug-in

According to different business scenarios, functional plug-ins can adapt to different UIs and animations. The following figure shows the styles of keyboard components in different question boards:

Figure 19: Keyboard assembly

There will be multiple atomic components and functional plug-ins in a question board component. Atomic components and functional plug-ins may rely on other atomic components, or they may rely on some general hooks, global context, animation, utils, etc. Atomic components and functional plug-ins are also sub-projects of multi-package projects. They can be independently exported and published as npm packages. They can be imported not only by any question board component, but also by other atomic components, and can also be imported into other projects.

General question board

In addition to the above atomic components and plug-ins, Ceramics also built-in general question board components to further improve development efficiency.

General question boards are some of the most basic question templates, such as general drag and drop, general selection, general fill in the blank, general connection, etc. The general question board encapsulates the basic logic and interaction, and more business logic is expressed in the form of data. This can improve the reusability of a single question template, making a general question board support N kinds of questions with high interaction and style similarity. Board demand.

Through the classification and abstraction of multiple question boards, we have summarized the basic elements of several types of basic question boards:

The multiple drag-and-drop questions in the following figure are composed of canvas, hot zone, draggable objects, and non-dragable objects.

Figure 20: Basic elements of the drag question

In the same way, multiple choice board include canvas, clickable objects, and non-clickable objects; blanks has canvas, objects, fill-in boxes, and option boxes; connection has canvas, connectable objects, and non-connectable objects . The types of objects include pictures, animations, texts, etc.

In addition to the basic elements, the feedback of the question board also has similarities: for example, the correct animation appears after the correct answer, the wrong animation appears after the wrong answer, the A object disappears and the B object is replaced when the connection is matched successfully or the drag is correct.

Consistent with the ECS design model mentioned above, when the object is given the interactive capabilities such as draggable, clickable, and connectable, and the corresponding feedback is added to each interaction, the basic interaction of a topic can be realized.

Based on this, we abstracted out a general question board, in which the element rendering, user interaction and submission judgment logic are processed, as for which elements on the page can be clicked, which can be dragged, the number, style, position and animation of each element , Feedback at different times, etc., can all be expressed with data, and the data drives the view for rendering.

We have encapsulated some basic classes, the more commonly used ones are Entity (object), Feedback (feedback), Area (hot zone), ShowCondition (time of appearance), etc.

All elements on the page belong to the Entity class. EntityType represents the type of object, including pictures, animations, text, titles, submit buttons, etc. Different types of objects have different configuration items and rendering methods.

Adding different attributes to the entity can give it different interactions, such as clickable for options that can be clicked, draggable for objects that can be dragged, droppable for hot spots that can be placed, and droppable hot spots can also be configured with shapes , Capacity, adsorption accuracy and other attributes.

export interface Entity extends Rectangle {
  id: string | number // 物体的唯一id
  name: string // 物体的名称
  entityType?: EntityType // 物体的类型
  draggable?: boolean // 可拖拽
  droppable?: boolean // 可放置
  clickable?: boolean // 可点击
  linkable?: boolean //可连线
  zIndex?: number // 层级，0-999，越大层级越高
  show?: ShowCondition // 何时显示，默认always
  feedbacks?: Feedback[]
  [key: string]: any
}

The feedbacks field is used to configure a group of feedback events, and feedback is mainly defined by timing and behavior.

export interface Feedback {
  type: FeedbackType | string // 反馈时机, FeedbackType和AND，OR组合而成的字符串
  action: FeedbackAction // 反馈的行为
  data: FeedbackData // 反馈行为需要的数据
}

The timing of feedback (types) are: click, put into hot zone, hot zone is full, drag to hot zone, drag wrong hot zone, start connection, end connection, animation playback completed, overall judgment question is correct, overall judgment question Errors etc.

The feedback actions include: displaying objects, hiding objects, playing audio, replacing pictures, checking answers, etc.

The same action may trigger multiple feedbacks, and one feedback may also be triggered at different trigger timings. Through Feedbacks, the feedback steps can be connected in series, which is similar to an animation script, making a conventional question board more abundant in gameplay.

After the above disassembly, the design idea of the general question board has gradually become clear. Take the general multiple choice question board in the following figure as an example:

Figure 21: General Multiple Choice

The core interactive process of multiple choice questions is:

1. Click on the pointable element to trigger the click feedback;
2. Meet the submission conditions, trigger the submission of feedback, proceed to judge the question, and trigger the positive/false feedback;
3. If the question end conditions are met, no answer is allowed at the end of the question, otherwise you can continue to answer.

There are three main timings for submitting answers:

1. When there is no submit button, click the clickable element to submit immediately;
2. When there is no submit button, the number of clicked elements is equal to the allowable number of submissions and submit immediately;
3. When there is a submit button, click the submit button to submit.

Multiple choice questions support single choice, multiple choice, partial choice, and choice without correct answer (such as voting). Therefore, the conditions for judging the answer are also different:

1. Single choice & multiple choice; the user's answer is consistent with the correct answer to the question;
2. Partial selection: the user's answer is partially consistent with the correct answer to the question;
3. Choices without correct answers: Any choice is correct.

The following is the basic data structure of the general selection:

interface QuestionConfig {
  canvas: Canvas //画布
  answer: Answer //正确答案
  entities: Entity[] //物体
  submit?: SubmitEntity //提交按钮
  audio?: { //音效
    correct?: string
    wrong?: string
    click?: string
  }
  aeTemplate?: string //动画模板
  maxCheckNum?: number //最大错误次数
  allowMultipleChoice?: boolean //是否允许多选
  allowPartChoice?: boolean // 是否允许部分选择
  showHint?: boolean // 是否出现引导
}

As mentioned above, the question is composed of question board and data. The main business logic in the general question board includes: processing the above rendering and interaction, defining the data structure of the configuration data, and performing corresponding display and feedback according to the configuration data. For example, rendering objects based on entities, binding clickable objects to clickable objects, displaying feedback based on the feedbacks field in the entity; processing submission logic and submit button rendering based on the submit field; judging topics based on fields such as answer, allowPartChoice, and allowMultipleChoice.

Through the rendering of data-driven pages, one question board can meet most of the same type of teaching and research needs. Teaching and research can configure data in the matching question board visual configuration interface to create new question boards without R&D intervention; R&D personnel can also base on The general question board is re-packaged. The general question board can be introduced as a component into other question boards. Multiple general question boards can be combined to form a new question board, which greatly reduces the development cost of the new question board.

3. Summary and Outlook

This article mainly introduces the technical implementation of Youdao interactive content engine Ceramics in terms of architecture design, user interaction, animation effects, component design, etc. It describes the production process of interactive content from the perspective of research and development, and how the interactive engine links the production of interactive content Operate more efficiently.

At present, Ceramics is still in the growth stage. In the future, we will continue to iterate and enrich the development tool chain. We will build a low-code development platform and build-up visual development question boards to make it easier to write interactive logic; open up the conversion channel from design drawings to code, develop material libraries, optimize UI and R&D cooperation experience; continuously improve efficient operation and rendering Ability to move towards gamification, support more complex gameplay, cooler animation effects; increase real-time interaction, multiplayer battle capabilities, and further enhance the interactivity and interest of scenes such as live lessons.

Our vision is to create a high-performance web interactive engine that allows front-end engineers to develop high-quality interactive content at a lower cost. In the future, we will continue to expand the influence of Ceramics and empower more businesses.