Vscode voice annotations make information richer (Part 1)

In this series I will share my complete process of making the "Voice Notes" plugin, or the phrase 'process' is more interesting than 'result'.

usage show

Download: voice-annotation

Configure + create a voice file storage address

Right click to call out 'Record Audio Notes'

When the recording is complete, click 'Save to Project'

Hover to play voice

background

When you start reading a piece of engineering code, you may occasionally read some unintelligible code. If it has comments but is still confused after reading it, or it does not write comments at all, then the most convenient way to solve this problem is The way is to ask the original developer directly, why did the classmate write this way?

Not only in the js language, but most languages already provide the syntax of "text comments", why do we still encounter the above problems? Suppose we have written comments in each piece of code but still can't understand it, let's have a look Which problems we can "mitigate" with voice plugins:

The writing is quite clear, and the incomprehension is indeed the reader's own problem ( can't solve it)
Wrote it but didn't understand it, the developer's own expressive ability ( can't be solved, between "Dark Effect" & "Curse of Knowledge" can't be self-aware)
Maybe it is related to a few requirements, it is very 'complex' to describe clearly in words, so I simply don't write it ( may be able to alleviate it)
I can't understand the form of the text. It is more common in our daily communication. It is not clear online. Solo face to face directly ( may be relieved)
Cross-page descriptions, such as expounding a large "debugging method", hiding which variable, forcibly assigning a variable, certain operations can achieve certain effects, and are often used in some scenarios of front-end mocking ( may be able to alleviate)

Students who have not done the vscode plugin recommend reading the introductory tutorial I wrote first:

`1. Analysis of function point 'technical solution'`

`① Identify specific comments`

We need to agree on a specific writing method, the plug-in recognizes this writing method as "voice annotation", and can broadcast correctly when the mouse hovers over it, currently I use the form of // voice_annotation + _number .

`② Play annotation audio`

Since it is written in vscode, in fact, the first choice is to use node to control the audio output device. If you open a new web page to play the sound, it will increase the user's operation link. I have investigated the existing audio plugins on the market, The main music player plugin is to open the web page, but we have no other operations here, such as fast forward loop playback and other slightly more complicated requirements, so we choose to use node to play.

`③ Record the annotation content`

If you want to eat eggs, of course you have to lay eggs. The recording function must be able to put the user's recording into the corresponding project, and return to the user's 'specific comment' writing method, assuming the return voice_annotation_20220201 , the user can directly paste it into the project and use it normally. There should not be many usage scenarios for this feature, so performance-related issues should be considered.

`④ Save audio information`

Most developers have more than one project on their computer, how can each project accurately play the voice of the corresponding project, and the generated audio file can be placed in the corresponding project after each recording, and can support the team to use.

`Second, the initial project, this time we use ts`

Install the vscode development tools globally, and initialize the project:

npm install -g yo generator-code 

yo code

( ts is really fragrant) The previous plug-ins were developed using native js. Since the plug-ins developed this time will use some 'uncommon' api , frankly, the documentation of vscode is not very friendly, so we directly look at the code ts type is the most convenient.

    "activationEvents": [
        "onStartupFinished"
    ],

Go to package.json file and modify the configuration of this plugin activation:

onStartupFinished stipulates that the plugin starts after the vscode initialization is completed, it will not occupy the performance when the user initializes the vscode, and our voice annotation is not the kind of program that is very urgent to start.

Go to the extension.ts file and clear all the useless codes

`before clearing`

`after clearing`

After executing yarn watch , start compiling the code, fn + F5 debugging code will report an error:

I was also confused when I saw the above error report. I didn't seem to have encountered it before using the native js write. I even upgraded the vscode editor and still reported an error, and then went to the code to check:

This is very clear, we just delineated a compatibility range, checked the official website log, and finally selected the minimum version around October 2020.

    "engines": {
        "vscode": "^1.50.0"
    },

Sure enough, when debugging again, a prompt box can appear:

pay attention: The @types/vscode below should also be changed to the same version number as above, otherwise it will not report an error at this stage, and an error will be reported when it is packaged and released.

  "dependencies": {
    "@types/vscode": "^1.50.0",
  }

`3. Identify special notes`

This function is to identify the "voice-annotation" on the page and highlight it. The reason why this function is developed first is that its own function is relatively independent, and the effect can be seen intuitively. When browsing the official website of vscode, I found two In this way, the effect of 'highlighting' a specific statement can be achieved.

Here we can predefine several regularities for recognizing "voice annotations". Suppose there is such a string of characters // voice_annotation_202202070612135678 , first we need to identify the // annotation character, and get the string of characters voice_annotation_202202070612135678 after '//' to assign it 'Style', and match the number 202202070612135678 in the content, so that the voice can be played accurately.

at src/util/index.ts

export const targetName = "voice_annotation"; // 单独拿出来方便后续的更改
export const getAllTargetReg = new RegExp(`//\\s(${targetName}_\\d+)`, "g");
export const testTargetReg = new RegExp(`${targetName}_(\\d+)`);

`4. Option 1: Custom Syntax Highlighting`

For example, the writing method function (){} in js is the syntax of the function, and in is a keyword. These writing methods with specific meaning will be given various 'highlights', then we can set // voice_annotation as a keyword,

`Step 1: Add the jojo scope to the package.json file`

It works on js , ts , tsx files.

  "contributes": {
    "grammars": [
      {
        "path": "./syntaxes/injection.json",
        "scopeName": "jojo.injection",
        "injectTo": [
          "source.js",
          "source.ts",
          "source.tsx"
        ]
      }
    ]
  },

`Step 2: Create injection.json file`

{
  "scopeName": "jojo.injection",
  "injectionSelector": "L:comment.line.double-slash",
  "patterns": [
    {
      "include": "#jojo"
    }
  ],
  "repository": {
    "jojo": {
      "match": "jojo_黄金体验",
      "name": "support.constant.color.0xFF00.css",
      "settings": {
        "foreground": "FFFFFF"
      }
    }
  }
}

scopeName defines the scope name as the name suggests.
injectionSelector defines the effective scope of the scope, which means that the content after // is effective.
jojo.match for keyword matching.
jojo.name is a bit special, it defines the style of this keyword, but it is not an ordinary css style, but needs to be defined in the file that defines the language style, and then it can only be used here.

This solution has many drawbacks, such as not being able to 'customize styles', changing settings dynamically, etc. In the end, I did not choose this implementation method.

`V. Option 2: Document Retrieval`

That is to get all the text content of the current 'document', and give it the ability to have a css-like style. This method is more flexible, and some interesting effects can be achieved through the pseudo-element.

First of all, we need to abstract a initVoiceAnnotationStyle method, which is specially used to set the style of voice-annotation , so that it is convenient to recognize "voice notes" anytime and anywhere:

extension.ts file for modification

import * as vscode from 'vscode';
import initVoiceAnnotationStyle from './initVoiceAnnotationStyle'; // 新增

export function activate(context: vscode.ExtensionContext) {
    initVoiceAnnotationStyle() // 新增
    context.subscriptions.push( // 新增
        vscode.window.onDidChangeActiveTextEditor(() => {
            initVoiceAnnotationStyle()
        })
    )
}

export function deactivate() { }

context.subscriptions.push is the registration command, here is a life cycle event registered.
vscode.window.onDidChangeActiveTextEditor listens to the event when we switch the development file in vscode, that is, every time we open the file, the reason why we use this is because our 'voice notes' will hardly be changed, and there is no need to occupy the performance of vscode.

`define style`

initVoiceAnnotationStyle file

import * as vscode from 'vscode';
import { getAllTargetReg } from './util/index'

export default function () {
    vscode.window.visibleTextEditors.map((res) => {
        const documentText = res.document.getText();
        const decorator = vscode.window.createTextEditorDecorationType({
            color: 'gray',
            cursor: "pointer",
            textDecoration: "underline",
            before: {
                contentText: "📢",
                width: "10px",
                height: '10px',
                margin: "0 8px 0 0"
            }
        })

        let n;
        const ranges = [];
        while (n = getAllTargetReg.exec(documentText)) {
            const startPos = res.document.positionAt(n.index + 3);
            const endPos = res.document.positionAt(n.index + 3 + n[1].length);
            const range = new vscode.Range(startPos, endPos)
            ranges.push(range);
        }
        res.setDecorations(decorator, ranges)
    })}

The knowledge points here are very interesting, let’s talk about it with the screenshots

vscode.window.visibleTextEditors You can understand it as getting the properties of the currently 'activated' code editing page, which includes the number of lines that are currently paged, and whether certain styles are activated or not.
res.document.getText() Get the content of the currently active page.
vscode.window.createTextEditorDecorationType creates a style, here only a few fixed styles are defined, but the great point is that 'pseudo elements' can be created here so that the gameplay is diverse, we can print its return result to see.
The key returned here is actually className .
ranges is used to record the start and end positions of voice-annotation .
while loop was chosen for processing because there may be more than one voice-annotation .
n.index + 3 is because // voice-annotation is preceded by // , which I chose to keep as it is.

For specific adjustment styles, we can use vscode developer tools

`6. Hover // voice_annotation playback style`

First of all, we need to define a new hover module, this module is responsible for the style after hover + play audio.

extension.ts Added:

import * as vscode from 'vscode';
import hover from './hover'; // 新增
import initVoiceAnnotationStyle from './initVoiceAnnotationStyle';

export function activate(context: vscode.ExtensionContext) {
    initVoiceAnnotationStyle()
    context.subscriptions.push(hover); // 新增
    context.subscriptions.push(
        vscode.window.onDidChangeActiveTextEditor(() => {
            initVoiceAnnotationStyle()
        })
    )
}

export function deactivate() { }

`define file content`

hover.ts

import * as vscode from 'vscode';
import { getVoiceAnnotationDirPath, targetName, testTargetReg } from './util'
let stopFn: () => void;

function playVoice(id: string) {
   // 这里是具体播放音频的逻辑...
}

export default vscode.languages.registerHoverProvider("*", {
    provideHover(documennt: vscode.TextDocument, position: vscode.Position) {
        stopFn?.()
        const word = documennt.getText(documennt.getWordRangeAtPosition(position));
        const testTargetRes = testTargetReg.exec(word);
        if (testTargetRes) {
            playVoice(testTargetRes[1])
            return new vscode.Hover('播放中 ...')
        }
    }
})

documennt.getText(documennt.getWordRangeAtPosition(position)) can get the current text content of hover .
testTargetReg.exec(word) Check whether the text of the current hover is the target text.
return new vscode.Hover('Playing...') The text returned here can only be in the format of markdown .
stopFn is the stop playback logic to be done later, because some audios may be very long, we don’t want to listen to half of the audio, or while listening to a certain audio, I hover over another voice note, then we Should stop the previous audio and play the current audio.

`end`

The next article will talk about how vscode plays sound, how audio files are transmitted and stored, and more pitfalls, wait for me to jump, this is the case, I hope to make progress with you.

Vscode voice annotations make information richer (Part 1)