Front-end screenshot implementation

头像
specialcoder
    阅读 15 分钟
    中文
    3

    Preface

    Demand background:

    • Goal: Turn the content of the page into a picture to share
    • Persistence: No server-side storage required
    • Client capabilities: does not rely on client capabilities

    Use puppeteer for screenshot service

    advantage:

    • The front end is simple to use, just call the interface
    • The sharing page can be separated from the content of the page displayed to the user, and there is no need to deal with issues such as style

    shortcoming:

    • New Service & Maintenance Service
    • Long interface response time will affect the experience

    in conclusion

    The survey found that the pure front end of html2canvas

    text

    How to produce a picture on the front end, the first thing that can be thought of is to use canvas to draw the area, and then turn it into a picture. However, manual canvas drawing has the following problems:

    • You cannot use ctx.drawImage to directly take a screenshot of the shared area: ctx.drawImage picture elements Image , svg elements, video elements Video , Canvas elements, ImageBitmap data, etc., but it cannot be drawn for other div or list elements li Drawn
    • Call canvas draw: layout calculation is required, drawing is also very cumbersome, resulting in a large amount of development
    • Need to solve some style issues/device compatibility issues

    For the above reasons, I plan to use the open source npm to realize the dom to image function.
    The following are the most used libraries:

    Library pros and cons analysis

    html-to-image

    /* ES6 */
import * as htmlToImage from 'html-to-image';
import { toPng, toJpeg, toBlob, toPixelData, toSvg } from 'html-to-image';

    Supported configurations:

    filter / backgroundColor / width, height / style / cacheBust / inagePlaceholder /  pixelRatio / fontEmbedCss

    Summarize:

    • Advantages: The library is small at 183 kB; it is relatively simple to use and generally does not require configuration
    • Disadvantages: There have been many compatibility problems in history; the src of the content in the screenshot area does not recognize base64; the clarity problem (it seems to be okay now)

    html2canvas

    Supported configuration: https://html2canvas.hertzen.com/configuration

    Summarize:

    • Advantages: 23K+ start volume, powerful functions, many solutions; no need to deal with clarity
    • Disadvantages: reduction degree problem; configuration burden; 2.96 MB

    About foreignObject:

    image.png

    html-to-image as used in the realization of foreignObject , it is necessary to browsers support this property. After looking at the level of support, in addition to IE, chrome safari support is quite good. So don't worry about not being able to use html-to-image because of this.

    Problems that html2canvas cannot solve [Avoided during development]:

    There are two important issues to pay attention to:

    • Picture problem: Support img tag picture, background picture, not border picture
    • Font problem: Font defined by FontFace is not supported

    Explanation in html2canvas:

    image.png

    Summarize

    • Conservative solution: use html2canvas directly, because it is powerful and compatible
    • The ultimate pursuit: You can use html-to-image first, because it is very small & can be called foolishly without worrying about the configuration & high degree of restoration. Use html2canvas for the bottom of the pocket.

    Scenario 1: The screenshot content cannot directly reuse the page content

    In the bottom of the page there is a sub-surface the Share button, then click on the picture to share a circle of friends, the bottom of the picture there is a small program carries a two-dimensional code path applet parameters, picture content and page content presented to the user inconsistent .

    Design

    Since the content of the picture is dynamic and consists of data, we need to draw it manually. The simple method that can be thought of is to the poster content in a div at and hide the div 16130848b3f3b7. Then you can perform overall operations on this div.

    Several elements invisible scheme
    display: none  ❎
visibility: hidden  【黑屏】
margin-left / nargin-top: -1000px  【黑屏】
position:absolute; left/top: -1000px 【黑屏】
z-index: -1000 ✅

    In addition to the invisible elements, in order to achieve the hidden effect without affecting the normal display of the page, it is also necessary to deal with so that the shared element 16130848b3f3f5 does not occupy the position of the document stream .

    Specific code

    html：

    <div id="app">
  <div class="wrapper">
    ...
    <div id="share" class="share-screen">
      ...
    </div>
  </div>
</div>

// style
.wrapper{
  position:relative;
}
.share-screen{
    width: 100%;
    z-index: -100; // 用户不可见
    position: absolute;
 // 定位
    top: 0;
    left:0;
    background-color: #fff;
}

    js:

      // 获取分享区域内容图片
  private getShareImg() {
    const node = document.getElementById('share');

    return html2canvas(node, {
      useCORS: true,
      x: 0,
      y: 0,
      logging: false,
    }).then(canvas => {
      return this.getPosterUrl(canvas);
    }).catch();
  }

  // 转成一张图片base64
  public getPosterUrl(canvas): string {
    return canvas.toDataURL('image/jpeg');
    // return canvas.toDataURL('image/jpeg').replace(/data:image\/\w+;base64,/, '');
  }

    Question reference

    Image cross-domain problem

    useCORS set to true

    Picture clarity issues

    After testing, there is no sharpness problem after html2canvas v1.0.01-rc7

    v1.0.01-rc5 before 06130848b3f564:

    /*
* 图片跨域及截图模糊处理
* https://www.cnblogs.com/padding1015/p/9225517.html 
*/
      let shareContent = domObj,//需要截图的包裹的（原生的）DOM 对象
          width = shareContent.clientWidth,//shareContent.offsetWidth; //获取dom 宽度
          height = shareContent.clientHeight,//shareContent.offsetHeight; //获取dom 高度
          canvas = document.createElement("canvas"), //创建一个canvas节点
          scale = 2; //定义任意放大倍数 支持小数
      canvas.width = width * scale; //定义canvas 宽度 * 缩放
      canvas.height = height * scale; //定义canvas高度 *缩放
      canvas.style.width = shareContent.clientWidth * scale + "px";
      canvas.style.height = shareContent.clientHeight * scale + "px";
      canvas.getContext("2d").scale(scale, scale); //获取context,设置scale
      let opts = {
          scale: scale, // 添加的scale 参数
          canvas: canvas, //自定义 canvas
          logging: false, //日志开关，便于查看html2canvas的内部执行流程
          width: width, //dom 原始宽度
          height: height,
          useCORS: true // 【重要】开启跨域配置
      };
html2canvas(shareContent,opts).then（）
    The content area of the screenshot is outside the visible area, and the html2canvas screenshot is blank

    Need to determine your screenshot area. Screenshot area style settings:

    // html
<div id="app">
  <div class="wrapper">
    ...
    <div id="share" class="share-screen">
      ...
    </div>
  </div>
</div>

// style
.wrapper{
  position:relative;
}
.share-screen{
    width: 100%;
    z-index: -100; // 用户不可见
    position: absolute;
 // 定位
    top: 0;
    left:0;
    background-color: #fff;
}

    html2canvas configuration:

    html2canvas(node, {
      useCORS: true, 
      x: 0, // x-offset:Crop canvas x-coordinate
      y: 0, // y-offset:Crop canvas y-coordinate
      logging: false,
}).then();
    Use html-to-image to report an error

    An error will be reported on Android:
    Error: Failed to execute 'toDataURL' on 'HTMLCanvasElement'......

    Please change to html2canvas .

    html2canvas image size adaptation problem

    The main problem here is dpr. The library will automatically match the device window.devicePixelRatio . After the above configuration, there will be no problems.

    html2canvas remote pictures [Allow cross-domain] occasionally does not show the problem

    html2canvas principle of 06130848b3f750 screenshots is node cloning and style cloning, so even if the pictures on the page are loaded, there is no guarantee that the pictures in the screenshots are normal. Remote image loading takes time, and this time is uncertain. If the image is converted to a picture before it is loaded, there will be a problem that the picture is not displayed.

    The solution is:

    • Loading local images will not have this problem
    • Remote pictures do not return the http address, they can return base64 encoding and be directly rendered by the browser, which is equivalent to returning the picture content directly. Or manually request the image address to convert the binary stream to base64.

    Possible difficulties

    There are a lot of pictures in the content of the page. Since the picture content needs to be requested during the screenshot process, a large number of picture requests may not guarantee the integrity of the screenshot. Even if there is no problem with the picture request, html2canvas may lose some pictures in the screenshot processing. (Reference: https://zhuanlan.zhihu.com/p/97099642)

    Therefore, the use of remote picture addresses is risky. After real machine testing, it is found that the probability of error is indeed high (50% is true).

    How to solve?

    • Convert to base64
    • Converted into blob, stored in memory using URL.createBlobURL

    Convert to base64

    This needs to discuss the advantages and disadvantages of base64.
    advantage:

    • Using base64 resources can reduce the number of network requests (usually remote pictures)
      The content of the image is straight out, and there will be no time-consuming or failure to load the image and cause the screenshot problem
    • toDataURL() has strong compatibility, except IE<=8 safari<=3 and basically other browsers support ( https://caniuse.com/?search=canvas.toDataUrl)
      shortcoming:
    • The size is 1/3 larger than the original picture
    • Pay attention to the format when converting, the transparent area of png → jpeg will turn into a black background (solution: https://evestorm.github.io/posts/46911/)

    Conversion:
    Create a dynamic image tag, the picture drawn on the canvas, using canvas.toDataURL export base64

      function toDataURL(url) {
    // 跳过base64图片
    const base64Pattern = /data:image\/(.)+;base64/i;
    if (base64Pattern.test(url)) {
      return Promise.resolve(url);
    }

    const fileType = getFileType(url) || 'png'; // 默认是png png支持透明

    return new Promise((resolve, reject) => {
      const canvas = document.createElement('canvas');
      const ctx = canvas.getContext('2d');
      const img = document.createElement('img');

      img.crossOrigin = 'Anonymous'; // 解决跨域图片问题，
      img.src = url;
      img.onload = () => {
          canvas.width = img.width;
          canvas.height = img.height;
          ctx.drawImage(img, 0, 0);
          const dataUrl = canvas.toDataURL(`image/${fileType}`);
          resolve(dataUrl);
      };
      img.onerror = e => {
        reject(e);
      };
    });
  }

  // 获取图片文件格式
  function getFileType(url) {
    const pattern = /.*\.(png|jpeg|gif)$/i; // Data URI scheme支持的image类型
    return (url.match(pattern) || [undefined, undefined])[1];
  }

    Convert to blob

    advantage:

    • Rich formats that support conversion, in addition to image, it can also support other types, no need to care about file type conversion issues

    shortcoming:

    Conversion:

      function toBlobURL(url) {
    return new Promise((resolve, reject) => {
      const canvas = document.createElement('canvas');
      const ctx = canvas.getContext('2d');
      const img = document.createElement('img');

      img.crossOrigin = 'Anonymous'; // 解决跨域图片问题，
      img.src = url;
      img.onload = () => {
          canvas.width = img.width;
          canvas.height = img.height;
          ctx.drawImage(img, 0, 0);
          // 关键
          canvas.toBlob(blob => {
             const blobURL = URL.createObjectURL(blob);
             resolve(blobURL);
            // 释放 blobURL
            URL.revokeObjectURL(blobURL)
         });;
      };
      img.onerror = e => {
        reject(e);
      };
    });
  }

    Realize that all pictures on the page src use base64

      const urlmap = {};

  // 获取图片文件格式
  function getFileType(url) {
    const pattern = /.*\.(png|jpeg|gif)$/i; // Data URI scheme支持的image类型
    return (url.match(pattern) || [undefined, undefined])[1];
  }

  // remoteURL转base64
  function toDataURL(url) {
    // 过滤重复值
    if (urlMap[url]) {
      return Promise.resolve(urlMap[url]);
    }

    // 跳过base64图片
    const base64Pattern = /data:image\/(.)+;base64/i;
    if (base64Pattern.test(url)) {
      return Promise.resolve(url);
    }

    const fileType = getFileType(url) || 'png'; // 默认是png png支持透明

    return new Promise((resolve, reject) => {
      const canvas = document.createElement('canvas');
      const ctx = canvas.getContext('2d');
      const img = document.createElement('img');

      img.crossOrigin = 'Anonymous'; // 解决跨域图片问题，
      img.src = url;
      img.onload = () => {
          canvas.width = img.width;
          canvas.height = img.height;

          ctx.drawImage(img, 0, 0);
          const dataUrl = canvas.toDataURL(`image/${fileType}`);
          resolve(dataUrl);
      };
      img.onerror = e => {
        reject(e);
      };
    });
  }

  // 批量转base64
  function convertToBlobImage(targetNode) {
    if (!targetNode) { return Promise.resolve(); }

    let nodeList = targetNode;
    if (targetNode instanceof Element) {
      if (targetNode.tagName.toLowerCase() === 'img') {
        nodeList = [targetNode];
      } else {
        nodeList = targetNode.getElementsByTagName('img');
      }
    } else if (!(nodeList instanceof Array) && !(nodeList instanceof NodeList)) {
      throw new Error('[convertToBlobImage] 必须是Element或NodeList类型');
    }

    if (nodeList.length === 0) {
      return Promise.resolve();
    }

    // 仅考虑<img>
    return new Promise(resolve => {
      let count = 0;
      // 逐一替换<img>资源地址
      for (let i = 0, len = nodeList.length; i < len; ++i) {
        const v = nodeList[i];
        let p = Promise.resolve();
        if (v.tagName.toLowerCase() === 'img') {
          p = toDataURL(v.src).then(blob => {
            v.src = blob;
          });
        }

        p.finally(() => {
          if (++count === nodeList.length) {
            return resolve(true);
          }
        });
      }
    });
  }

    Precautions:

    • When the remote image is png, it should be converted to png format ( toDataURL(image/png )), because png images support transparency, the alpha channel will be turned off when converted to jpeg files, causing the transparent area to become a black background. Recommended adaptive file type, set to png by default
    • Image types supported by Data URI scheme: png | jpeg | gif
    • When drawing a picture on the canvas, an error may be reported: "Failed to execute 'toBlob' on 'HTMLCanvasElement': Tainted canvases may not be exported" . The reason is that the CORS picture will pollute the canvas and make it impossible to read its data. Add the following code to solve the cross-domain picture problem: img.crossOrigin = 'Anonymous'

    Expand knowledge

    <img /> src the support of three kinds: url/base64/blob .

    1. about several commonly used web image formats: gif, jpg, png, webp

    image.png

    2. Process arrayBuffer files
    // buffer -> base64
function getData(){
  let binary = '';
  const bytes = new Uint8Array(buffer);
  const len = bytes.byteLength;
  for (let i = 0; i < len; i++) {
   binary += String.fromCharCode(bytes[i]);
  }
  return 'data:image/png;base64,' + btoa(binary); // window.btoa() 方法用于创建一个 base-64 编码的字符串。
}
    3. base64 to blob:
    // 原理：利用URL.createObjectURL为blob对象创建临时的URL
function base64ToBlob ({b64data = '', contentType = '', sliceSize = 512} = {}) {
  return new Promise((resolve, reject) => {
    // 使用 atob() 方法将数据解码
    let byteCharacters = atob(b64data);
    
let byteArrays = [];
    for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
        let slice = byteCharacters.slice(offset, offset + sliceSize);
        let byteNumbers = [];
        for (let i = 0; i < slice.length; i++) {
            byteNumbers.push(slice.charCodeAt(i));
        }
        // 8 位无符号整数值的类型化数组。内容将初始化为 0。
        // 如果无法分配请求数目的字节，则将引发异常。
        byteArrays.push(newUint8Array(byteNumbers));
      }
      let result = new Blob(byteArrays, {
        type: contentType
      })
      result = Object.assign(result,{
        // jartto: 这里一定要处理一下 URL.createObjectURL
        preview: URL.createObjectURL(result),
        name: `图片示例.png`
      });
      resolve(result)
    })
 }
    4. Blob to base64:

    Compatibility: IE>9 safari>5 https://caniuse.com/?search=readAsDataURL 

    // 原理：利用fileReader的readAsDataURL，将blob转为base64
function blobToBase64(blob) {
  return new Promise((resolve, reject) => {
      const fileReader = new FileReader();
      fileReader.onload = (e) => {
        resolve(e.target.result);
      };
      // readAsDataURL
      fileReader.readAsDataURL(blob);
      fileReader.onerror = () => {
        reject(newError('文件流异常'));
      };
    });
}

    Scenario 2: The screenshot content is an extension of the page content

    The main content of the screenshot is the content displayed on the page, and the diversion banner is spliced at the bottom.

    How to realize dynamic splicing

    You can use html2canvas of onclone property, copy the contents of the node that implements out of the changes. Modifying elements in onclone will not affect the original elements of the page.

    Leave the dynamic display content out of the document flow and make it invisible:
    <!-- html -->
<div class="wrapper">
  <!-- 主要内容 -->
  <div class="content" id="share"></div>
  <!-- 动态添加的内容 -->
  <div class="dynamic-contetn" id="share-footer"></div>
</div>
    // css
.wrapper{
  position:relative;
}
.dynamic-content{
   
position: absolute;
  top:0;
  left:0;
  right: 0;
  z-index:-100;
}
    When taking a screenshot, modify it in onclone to make the dynamic content appear in the normal position and make it visible:
      const node = document.getElementById('share'); // 分享海报的主要内容区域
  const 
shareFooter = document.getElementById('share-footer'); // 海报底部内容
  html2canvas(node,{
      onclone: (doc: Document) => {
        // 将海报底部内容区域显示出来，用于和内容一起截屏
        shareFooterNode.style.position = 'relative';
        shareFooterNode.style.display = 'flex';
        shareFooterNode.style.zIndex = '1';
      }
   }).then();
    Control the screenshot size:

    Because the height of the screenshot is the offsetHeight of the Node element by default, now this height should become Node height + dynamic area height :

        const node = document.getElementById('share'); // 分享海报的主要内容区域
    const shareFooter = document.getElementById('share-footer'); // 海报底部内容
    const { height: footerHeight} = shareFooter.getBoundingClientRect();

    html2canvas(node, {
        useCORS: true,
        logging: false,
        height: node.offsetHeight + footerHeight,
       }).then();

    Another practice of intercepting the content of the visible area

    Add a shell to the child element of the body, so that the web page content scrolls in the shell.

    html:

    <html>
 <body>
   <div class="wrapper">
     <div id="share" class="content"></div>
   </div>
 </body>
</html>
    // css

.wrapper{
  position:fixed;
  left:0;
  right:0;
  top:0;
  bottom:0;
}

    javascript:

        const wrapper = document.getElementsByClassName('wrapper')[0]; // wrapper 用于监听scroll
    const node = document.getElementById('share'); // 分享海报的主要内容区域

    const { scrollTop } = wrapper;
    const { height: footerHeight} = shareFooter.getBoundingClientRect();

    return this.convertToBlobImage(node).then(res => {
      return html2canvas(node, {
        useCORS: true,
        logging: false,
        x: 0,
        y: -scrollTop, // 重要
        height: node.offsetHeight, // 重要
      }).then();

    This will also allow normal screenshots.
    So the setting of x/y is related to your layout. You can try all of them and use the one with normal screenshots.

    Summary in two scenarios

    The screenshot content is completely different from the page content:

    • Prepare a screenshot area node content and hide it
    • Use x:0 y:0 to solve the problem of screenshot area

    The screenshot content is consistent with the large content of the page:

    • Reuse page node content
    • Use the shell under the body and y:-scrollTop solve the problem of the screenshot area
    • If there is dynamic content, use height: node.offsetHeight + dynamicHeight

    The image should be converted to base64 content to ensure the integrity of the screenshot

    Two major problems in special scenarios

    A canvas size of 06130848b4018d is too large, causing canvas.toDataURL() to return to data:/

    Solution:
    detects whether it is legal. If it is illegal, lower the html2canvas scale (which will affect the definition). If it falls to 1 and it is not legal, there is no way.

    There are no restrictions on each browser, and there is no API directly used for detection. A detection library canvas-size recommended. Start testing from the current dpr of the device, backward compatible:

      private async getAvailableScale(node) {
    let scale = 1;

    const {offsetWidth, offsetHeight} = node;
    const nodeWidth = Math.ceil(offsetWidth);
    const nodeHeight = Math.ceil(offsetHeight);
    const dpr = window.devicePixelRatio;

    const sizes = [];
    for (let i = dpr; i >= 1; i--) {
      sizes.push([nodeWidth * dpr, nodeHeight * dpr]);
    }

    try {
      const { width, height, benchmark } = await canvasSize.test({
        sizes,
        usePromise: true,
        useWorker : true,
      });
      console.log(`Success: ${width} x ${height} (${benchmark} ms)`);
      scale =  width / nodeWidth;
    } catch ({ width, height, benchmark }) {
      console.log(`Error: ${width} x ${height} (${benchmark} ms)`);
    }
    return scale;
  }

    Two WeChat SDK share the 512K limit problem of Moments of Friends

    solution:
    Using the second parameter of canvas.toDataURL(type,quality), compressing the image will reduce the image size (but at the same time the image quality will also decrease). quality: (0,1) , the default is 0.92

     public getPosterUrl(canvas): string {
    const img = canvas.toDataURL('image/jpeg');
    const size =  Math.ceil(getImgSize(img) / 1024); // 获取base64图片大小
    console.log('海报图片大小(kb)：', size);
    console.log('海报图片质量：', 400 / size); // 超出取值范围0-1，将会使用默认值 0.92
    // HuaWei mate20 / android10 长图下不压缩会分享失败
   // 400kb是基于512kb经过测试算出来的一个比较好的参考值
    return canvas.toDataURL('image/jpeg', 400 / size).replace(/data:image\/\w+;base64,/, '');
  }

    dom-to-image principle

    Maybe someday there will be a simple, standard way of exporting a part of HTML to an image (perhaps it already exists?) (then this script becomes proof of all the problems I have experienced in order to achieve this goal), but to I haven't found it so far.

    The library uses SVG functionality, which allows arbitrary HTML content to be included <foreignObject> Therefore, in order to present the DOM node, the following steps will be performed:

    1. Recursively clone the original DOM node

    2. Calculate the style of the node and each child node, then copy it to the corresponding clone

      • And don’t forget to recreate the pseudo-elements, because of course they won’t be cloned in any way

    3. Embed web fonts

      • @font-face statements that may represent web fonts
      • Parse the file URL and download the corresponding file
      • base64 encode and inline content as dataURLs
      • Connect all the processed CSS rules and put them in a <style> element, and then append it to the clone

    4. Embed image

      • Embed the image URL in the <img> element
      • Embedded images used in background CSS properties, in a manner similar to fonts
    5. Serialize the cloned node to XML
    6. Wrap the XML into <foreignObject> tags, then wrap it in SVG, and then make it a data URL
    7. (Optional) To get the PNG content or raw pixel data in the form of Uint8Array, please create a Image SVG as the source, and present it on the off-screen canvas you also created, and then read the content from the canvas
    8. complete

    Reference

    postscript

    There will be new scenes and problems accumulated in the follow-up and will continue to be supplemented.

    webhtml5html2canvas
    specialcoder
    2.2k 声望170 粉丝

    前端 设计 摄影 文学

    « 上一篇
    移动端播放器
    下一篇 »
    微信开发手册

    引用和评论

    推荐阅读
    头像
    认识包管理工具： npm、yarn和pnpm

    specialcoder赞 3阅读 762

    头像
    JS校验身份证号

    寒青赞 10阅读 4.5k评论 2

    头像
    Jerry和您聊聊Chrome开发者工具

    注销赞 9阅读 4.8k评论 3

    头像
    面向企业级前端应用的开发框架 UI5 的发展简史介绍

    注销赞 5阅读 7.8k

    头像
    浏览器性能优化

    寒青赞 9阅读 3.8k

    头像
    startViewTransition的简单示例

    热饭班长赞 1阅读 3.7k

    头像
    【翻译】第三章 - [Dan_Ristic]webrtc开发实践

    zeronlee赞 2阅读 2.5k

    0 条评论
    得票最新