I use 10,000 pictures to synthesize our beautiful moments

The moon shines back on the lake, wild cranes rush towards idle clouds

Preface

Yesterday was Valentine's Day. Compared to everyone, I spent the holiday very happily~ Me too😚

Okay, let's not talk nonsense. Today is a very interesting project for everyone. By cutting the target image, 10,000 squares are obtained, and the image we have selected and the corresponding filling square are used to achieve a thousand-image imaging effect. You can use it to put together any meaningful big picture you want to put together. (For example, I want to use it to make a super super super super super super super large wedding photo of all the photos taken from my relationship with the object to the wedding. The grass in my hometown Poyang Lake is paved, and I used a drone to overlook it from a high altitude. It is very interesting~ I will bury a point here, I hope it will be realized in a few years 😊)

First of all, this article is fabric introductory article . It is also used by myself to practice and summarize. I will share it with everyone here and grow together!

Into the title

First, we initialize an 800*800 canvas

(The style of the interface, I will not express too much here, we mainly talk about the realization of logic functions)

//初始化画布
initCanvas() {
    this.canvas = new fabric.Canvas("canvas", {
        selectable: false,
        selection: false,
        hoverCursor: "pointer",
    });
    this.ctx = canvas.getContext("2d");
    this.addCanvasEvent();//给画布添加事件
},

Customize the size of the canvas according to the configuration of your computer. At present, I have not found a way to do similar imaging on the web side. Implementing this function on the web side is indeed very performance-consuming, because the amount of data that needs to be processed is so large and the amount of calculation is Also big
needs to pay attention to: the 800*800 canvas has ctx.getImageData pixels, and each pixel obtained through 0611e1ca61c23d is 4 values, which is 2,560,000 values. We need to deal with these 2560000 values later, so I won't make them bigger here.

Use fabric to draw the target picture

to note is that we draw to the canvas by local images, you need to get the file by file window.URL.createObjectURL(file) the type of document into blob url

Like you like to use the upload component of elementUI, you just write

//目标图片选择回调
slectFile(file, fileList) {
    let tempUrl = window.URL.createObjectURL(file.raw);
    this.drawImage(tempUrl);
},

I don't like its components here, because when selecting resource pictures later, there will be a file list when selecting thousands of pictures, and I don't want to hide it (mainly I want to share the custom file selection)
So i wrote it like this

export function inputFile() {
    return new Promise(function (resolve, reject) {
        if (document.getElementById("myInput")) {
            let inputFile = document.getElementById("myInput");
            inputFile.onchange = (e) => {
                let urlArr = [];
                for (let i = 0; i < e.target.files.length; i++) {
                    urlArr.push(URL.createObjectURL(e.target.files[i]));
                }
                resolve(urlArr);
            };
            inputFile.click();
        } else {
            let inputFile = document.createElement("input");
            inputFile.setAttribute("id", "myInput");
            inputFile.setAttribute("type", "file");
            inputFile.setAttribute("accept", "image/*");
            inputFile.setAttribute("name", "file");
            inputFile.setAttribute("multiple", "multiple");
            inputFile.setAttribute("style", "display: none");
            inputFile.onchange = (e) => {
                // console.log(e.target.files[0]);
                // console.log(e.target.files);
                // let tempUrl = URL.createObjectURL(e.target.files[0]);
                // console.log(tempUrl);
                let urlArr = [];
                for (let i = 0; i < e.target.files.length; i++) {
                    urlArr.push(URL.createObjectURL(e.target.files[i]));
                }
                resolve(urlArr);
            };
            document.body.appendChild(inputFile);
            inputFile.click();
        }
    });
}

After getting the file through the above method, I have converted the image file into the URL of the blob for us to use (note that the file selection is asynchronous, so here needs to be written with promise)

//绘制目标图片
drawImage(url) {
    fabric.Image.fromURL(url, (img) => {
        //设置缩放比例,长图的缩放比为this.canvas.width / img.width,宽图的缩放比为this.canvas.height / img.height
        let scale =
            img.height > img.width
                ? this.canvas.width / img.width
                : this.canvas.height / img.height;
        img.set({
            left: this.canvas.height / 2, //距离左边的距离
            originX: "center", //图片在原点的对齐方式
            top: 0,
            scaleX: scale, //横向缩放
            scaleY: scale, //纵向缩放
            selectable: false, //可交互
        });
        //图片添加到画布的回调函数
        img.on("added", (e) => {
            //这里有个问题,added后获取的是之前的画布像素数据,其他手动触发的事件,不会有这种问题
            //故用一个异步解决
            setTimeout(() => {
                this.getCanvasData();
            }, 500);
        });
        this.canvas.add(img); //将图片添加到画布
        this.drawLine(); //绘制网格线条
    });
},

After drawing the picture, draw a 100*100 grid on the canvas by the way

//栅格线
drawLine() {
    const blockPixel = 8;
    for (let i = 0; i <= this.canvas.width / blockPixel; i++) {
        this.canvas.add(
            new fabric.Line([i * blockPixel, 0, i * blockPixel, this.canvas.height], {
                left: i * blockPixel,
                stroke: "gray",
                selectable: false, //是否可被选中
            })
        );
        this.canvas.add(
            new fabric.Line([0, i * blockPixel, this.canvas.height, i * blockPixel], {
                top: i * blockPixel,
                stroke: "gray",
                selectable: false, //是否可被选中
            })
        );
    }
},

drawn, you can see the effect of adding grid lines to the picture, which is still pretty good~😘

Store the image color in blocks in an array

I wrote this at the beginning and the browser

---

I cry 😥, this is too much loop nesting (and the base is 800*800*4==2560000-->It must be written well, otherwise I am sorry that pixelList has been manipulated 2560000 times by me) I have to optimize the writing, since it is browsed The device is exploded, the stupid method doesn't work, it can only be changed~

First, let us explain that the length and width of each small block here is 8 pixels (the smaller the smaller, the finer the precision of the synthesized picture, the larger and the more blurred)

//获取画布像素数据
getCanvasData() {
    for (let Y = 0; Y < this.canvas.height / 8; Y++) {
        for (let X = 0; X < this.canvas.width / 8; X++) {
            //每8*8像素的一块区域一组
            let tempColorData = this.ctx.getImageData(X * 8, Y * 8, 8, 8).data;
            //将获取到数据每4个一组,每组都是一个像素
            this.blockList[Y * 100 + X] = { position: [X, Y], color: [] };
            for (let i = 0; i < tempColorData.length; i += 4) {
                this.blockList[Y * 100 + X].color.push([
                    tempColorData[i],
                    tempColorData[i + 1],
                    tempColorData[i + 2],
                    tempColorData[i + 3],
                ]);
            }
        }
    }
    console.log(mostBlockColor(this.blockList));
    this.mostBlockColor(this.blockList);//获取每个小块的主色调
    this.loading = false;
},

changing the way of writing, here we divide each 8*8 pixel block into a group to get 10,000 elements, and each element has 4 values, which represent the value of RGBA. Later we will Fill the corresponding pixel block

getting all the pixel values on the canvas, we need to
Later, we need to determine which picture the square is filled with by calculating the color difference between the main color of these small squares and the resource image.
I am very excited here, I feel that it is almost half done, but it is not the case, I will scratch my scalp later 😭

 //获取每个格子的主色调
mostBlockColor(blockList) {
    for (let i = 0; i < blockList.length; i++) {
        let colorList = [];
        let rgbaStr = "";
        for (let k = 0; k < blockList[k].color.length; k++) {
            rgbaStr = blockList[i].color[k];
            if (rgbaStr in colorList) {
                ++colorList[rgbaStr];
            } else {
                colorList[rgbaStr] = 1;
            }
        }
        let arr = [];
        for (let prop in colorList) {
            arr.push({
                // 如果只获取rgb,则为`rgb(${prop})`
                color: prop.split(","),
                // color: `rgba(${prop})`,
                count: colorList[prop],
            });
        }
        // 数组排序
        arr.sort((a, b) => {
            return b.count - a.count;
        });
        arr[0].position = blockList[i].position;
        this.blockMainColors.push(arr[0]);
    }
    console.log(this.blockMainColors);
},

seeds are not good, the draft paper is used

Get the main color of each resource map

export function getMostColor(imgUrl) {
    return new Promise((resolve, reject) => {
        try {
            const canvas = document.createElement("canvas");
            //设置canvas的宽高都为20,越小越快,但是越小越不精确
            canvas.width = 20;
            canvas.height = 20;
            const img = new Image(); // 创建img元素
            img.src = imgUrl; // 设置图片源地址
            img.onload = () => {
                const ctx = canvas.getContext("2d");
                const scaleH = canvas.height / img.height;
                img.height = canvas.height;
                img.width = img.width * scaleH;
                ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
                console.log(img.width, img.height);
                // 获取像素数据
                let pixelData = ctx.getImageData(0, 0, canvas.width, canvas.height).data;
                let colorList = [];
                let color = [];
                let colorKey = "";
                let colorArr = [];
                // 分组循环
                for (let i = 0; i < pixelData.length; i += 4) {
                    color[0] = pixelData[i];
                    color[1] = pixelData[i + 1];
                    color[2] = pixelData[i + 2];
                    color[3] = pixelData[i + 3];
                    colorKey = color.join(",");
                    if (colorKey in colorList) {
                        ++colorList[colorKey];
                    } else {
                        colorList[colorKey] = 1;
                    }
                }
                for (let prop in colorList) {
                    colorArr.push({
                        color: prop.split(","),
                        count: colorList[prop],
                    });
                }
                // 对所有颜色数组排序,取第一个为主色调
                colorArr.sort((a, b) => {
                    return b.count - a.count;
                });
                colorArr[0].url = imgUrl;
                console.log(
                    `%c rgba(${colorArr[0].color.join(",")})`,
                    `background: rgba(${colorArr[0].color.join(",")})`
                );
                resolve(colorArr[0]);
            };
        } catch (e) {
            reject(e);
        }
    });
}

we randomly select some files, print out their main color to see the effect

Color space

To request the color difference of colors, we first need to understand the definition of colors together. There are many ways to represent colors, and their standards are different, such as CMYK, RGB, HSB, LAB, etc...
Here we are RGBA, which is the RGB color model with additional Alpha information

RGBA is the color space representing Red, Green, Blue and Alpha. Although it is sometimes described as a color space, it is actually just an RGB model with additional information. The color used is RGB, which can belong to any RGB color space, but Catmull and Smith proposed this indispensable alpha value between 1971 and 1972, making alpha rendering and alpha composition possible. The author’s name for alpha is derived from the classical linear interpolation equation αA + (1-α)B, which uses this Greek letter.

You can see the related introduction of other colors
here: https://zhuanlan.zhihu.com/p/24281841
Or here https://baike.baidu.com/item/%E9%A2%9C%E8%89%B2%E7%A9%BA%E9%97%B4/10834848?fr=aladdin

How to find the color difference

Since the distribution of colors in space is as shown in the introduction above, here we use the Euclidean distance method learned in middle school to find the absolute distance of two colors, and the degree of similarity of the two colors can be known by their distance.

First we understand the basic concept of Euclidean distance

Euclidean metric (also known as Euclidean distance) is a commonly used distance definition, which refers to the true distance between two points in the m-dimensional space, or the natural length of the vector (that is, the point to the origin the distance). The Euclidean distance in two-dimensional and three-dimensional space is the actual distance between two points.

converts the formula into a code:

//计算颜色差异
colorDiff(color1, color2) {
    let distance = 0;//初始化距离
    for (let i = 0; i < color1.length; i++) {
        distance += (color1[i] - color2[i]) ** 2;//对两组颜色r,g,b[a]的差的平方求和
    }
    return Math.sqrt(distance);//开平方后得到两个颜色在色彩空间的绝对距离
},

There are many ways to calculate the color difference, see wikiwand: https://www.wikiwand.com/en/Color_difference#/sRGB
Or you can use color processing libraries like ColorRNA.js for comparison, we won't describe too much here.

Render the picture after calculating the difference

Here we need to compare the main color of each pixel block with the main color of all resource images, and take the one with the smallest difference and render it on the corresponding square

//生成图片
generateImg() {
    this.loading = true;
    let diffColorList = [];
    //遍历所有方块
    for (let i = 0; i < this.blockMainColors.length; i++) {
        diffColorList[i] = { diffs: [] };
        //遍历所有图片
        for (let j = 0; j < this.imgList.length; j++) {
            diffColorList[i].diffs.push({
                url: this.imgList[j].url,
                diff: this.colorDiff(this.blockMainColors[i].color, this.imgList[j].color),
                color: this.imgList[j].color,
            });
        }
        //对比较过的图片进行排序,差异最小的放最前面
        diffColorList[i].diffs.sort((a, b) => {
            return a.diff - b.diff;
        });
        //取第0个图片信息
        diffColorList[i].url = diffColorList[i].diffs[0].url;
        diffColorList[i].position = this.blockMainColors[i].position;
        diffColorList[i].Acolor = this.blockMainColors[i].color;
        diffColorList[i].Bcolor = diffColorList[i].diffs[0].color;
    }
    this.loading = false;
    console.log(diffColorList);
    //便利每一个方块,对其渲染
    diffColorList.forEach((item) => {
        fabric.Image.fromURL(item.url, (img) => {
            let scale = img.height > img.width ? 8 / img.width : 8 / img.height;
            // img.scale(8 / img.height);
            img.set({
                left: item.position[0] * 8,
                top: item.position[1] * 8,
                originX: "center",
                scaleX: scale,
                scaleY: scale,
            });
            this.canvas.add(img);
        });
    });
},

Good guy!!! What is this??? This is all night, and this is out?

five o'clock, I haven't slept yet~ 1611e1ca61c713

Don't give up, don't give up, persist to the end is victory

Carefully analyze each of the next steps and find the problem step by step
The correctness of the pixel data is seen from the pixel data of the target image at the beginning, but the problem is not found, and the data is okay. The preliminary judgment is that there is a problem with the main color of the pixel block, so I thought about whether it will be the main color. Instead of taking the color that appears the most times in a picture or a pixel block as the main color, but taking the average of all their colors as the main color?
Thinking of this, I am very excited!
Almost awakened the sleeping melon baby, I started to comb

Here, I changed each 8*8 small square to

//获取每个格子的主色调
mostBlockColor(blockList) {
    for (let i = 0; i < blockList.length; i++) {
        let r = 0,
            g = 0,
            b = 0,
            a = 0;
        for (let j = 0; j < blockList[i].color[j].length; j++) {
            r += blockList[i].color[j][0];
            g += blockList[i].color[j][1];
            b += blockList[i].color[j][2];
            a += blockList[i].color[j][3];
        }
        // 求取平均值
        r /= blockList[i].color[0].length;
        g /= blockList[i].color[0].length;
        b /= blockList[i].color[0].length;
        a /= blockList[i].color[0].length;
        // 将最终的值取整
        r = Math.round(r);
        g = Math.round(g);
        b = Math.round(b);
        a = Math.round(a);
        this.blockMainColors.push({
            position: blockList[i].position,
            color: [r, g, b, a],
        });
    }
    console.log(this.blockMainColors);
}

Then, for each picture, it is also changed to find the main color by the average value

export function getAverageColor(imgUrl) {
    return new Promise((resolve, reject) => {
        try {
            const canvas = document.createElement("canvas");
            //设置canvas的宽高都为20,越小越快,但是越小越不精确
            canvas.width = 20;
            canvas.height = 20;
            const img = new Image(); // 创建img元素
            img.src = imgUrl; // 设置图片源地址
            img.onload = () => {
                console.log(img.width, img.height);
                let ctx = canvas.getContext("2d");
                const scaleH = canvas.height / img.height;
                img.height = canvas.height;
                img.width = img.width * scaleH;
                ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
                // 获取像素数据
                let data = ctx.getImageData(0, 0, canvas.width, canvas.height).data;
                let r = 0,
                    g = 0,
                    b = 0,
                    a = 0;
                // 取所有像素的平均值
                for (let row = 0; row < canvas.height; row++) {
                    for (let col = 0; col < canvas.width; col++) {
                        r += data[(canvas.width * row + col) * 4];
                        g += data[(canvas.width * row + col) * 4 + 1];
                        b += data[(canvas.width * row + col) * 4 + 2];
                        a += data[(canvas.width * row + col) * 4 + 3];
                    }
                }
                // 求取平均值
                r /= canvas.width * canvas.height;
                g /= canvas.width * canvas.height;
                b /= canvas.width * canvas.height;
                a /= canvas.width * canvas.height;

                // 将最终的值取整
                r = Math.round(r);
                g = Math.round(g);
                b = Math.round(b);
                a = Math.round(a);
                console.log(
                    `%c ${"rgba(" + r + "," + g + "," + b + "," + a + ")"}
                                                                        `,
                    `background: ${"rgba(" + r + "," + g + "," + b + "," + a + ")"};`
                );
                resolve({ color: [r, g, b, a], url: imgUrl });
            };
        } catch (e) {
            reject(e);
        }
    });
}

exciting time has come!!!!!!!!!!!! Ahhhhhhhhh!! I am so excited, the victory is right in front of my eyes, and I have a jor!

After a meal operation, select the target picture, select the resource picture, and click the Generate Picture button, I began to wait for the call of victory!

I go, it's even uglier, what's

Immediately afterwards, I became enthusiastic, and I was very enthusiastic when encountering such challenging things.
So I started to analyze the small main colors that I processed, and I found that they seemed to be regular

I think what is affecting it, the picture can't be drawn in the same color, what is the same color???

wo kao~ will not be the 100*100 line I drew

So I went back to the drawLine function, I commented it out~

nice!

Each square can be stretched, rotated, moved interactively, and the basic functions of the canvas here are over~Sahua~🌹🏵🌸💐🌺🌻🌼🌷

We can also export the generated pictures. The good friends of the machine can define a large canvas, or number the pictures, and print them out. It is that can be used to make huge composite pictures (such as my previous The mentioned wedding photos, etc., are still very interesting)

 //导出图片
exportCanvas() {
    const dataURL = this.canvas.toDataURL({
        width: this.canvas.width,
        height: this.canvas.height,
        left: 0,
        top: 0,
        format: "png",
    });
    const link = document.createElement("a");
    link.download = "canvas.png";
    link.href = dataURL;
    document.body.appendChild(link);
    link.click();
    document.body.removeChild(link);
},

This Valentine’s Day is a bit fulfilling, it’s half past six in the morning~ I have a wave of liver, sleeping, life is important, and I have to go out to play during the day😅😅

at last

sublimate:

In the romantic Tanabata, there is even a smell of love in the air. Inviting lovers to each other happily, after dusk, the willow heads, whispering, the beautiful day, the full moon is good! Bless all lovers, happiness!

I put this project on my github ( https://github.com/wangrongding), like friends, remember to like it~

I am very happy to be strong with everyone! You can follow my official account, Qianpuzhai. I have formed a front-end technology exchange group. If you want to communicate and learn with like-minded friends, you can also add my personal WeChat (ChicSparrow), I will pull you into the group and cheer together! I am Rongding, join me in the key Caps and characters jump horizontally, shuttle between codes and programs. 🦄