1 Effect

• Static site:
  • blog.makergyt.com
    
  • Alternate link: github.blog.makergyt.com
    
• Applet:
  

  
  
• Sparrow: <MakerGYT blog>

2 Demand Analysis

2.1 Do

• Focus on writing markdown documents, or on content.
• One-end writing, multi-end synchronization: applets, static sites
• Fast static hosting, continuous integration, page resource loading

2.2 Do not

• You don't need to be satisfied with writing anytime, anywhere, because most of the posts you write anytime, anywhere are casual or recorded. If you want them to appear, they must be sorted out.
• There is no need to customize the theme style. Blogs don't have a strong personality in terms of the main business types (except comments, favorites, collections)

3 System Design

3.1 Overview Design

3.1.1 Architecture Design

The main idea is to publish git-managed articles (markdown type) to applets and static sites (framework for building md documents such as hexo, jeklly, and so on).
Technical route:

1. Update=>Source-side CURD operation via git
2. Resolve=>Resolve md to html through serverless
3. Synchronization=>Build and deploy to object storage (static hosting) through CI; Update git repository to cloud storage via webhook (applet)
4. Browse=> Refresh CDN by triggering cloud function via object storage (static hosting); Resolve HTML (applet) through the rich-text component

3.1.2 Technology Selection and Development Framework

On the development framework, since it was originally developed for WeChat applets and may have unknown problems, it uses native development instead of multiterminal or other precompiled frameworks. On the applet UI, reference but do not rely on the WeUI component library, as encapsulating unnecessary features may cause redundancy in the code package.

type	programme	Remarks
Code Managed	Coding	github api access rate is slow and unstable
Cloud Development	Tencent Cloud TCB	Cloud Development Service with Applets
Continuous Integration	Coding CI	Defining pipeline s using Jenkinsfile
Static Managed	Tencent Cloud COS	Also available Aliyun OSS Or directly using cloud development Static Site Hosting , use object storage with content distribution acceleration.
Markdown parsing	markdown-it	markdjs is also available, but markdown-it supports extensions
Rich Text Rendering	parser	Richer and more stable than native rich-text

3.1.3 Interface Design

Because it is a content-based application, special attention should be paid to the visual specifications so that users can get a better reading experience. The following specifications are referenced WEDESIGN and Ant Design And has been modified and supplemented according to actual needs.
Typeface:

Font size pt	Pixel px	colour	purpose
17	17	#000000	Top level information on page, list title
17	17	#B2B2B2	Timestamp and form defaults
14	14	#888888	Secondary descriptions on the page, with list titles
14	14	#353535	Bulk Text
13	13	#576b95	Page ancillary information, weakened content such as links
13	13	#09bb07	Complete Word
13	13	#e64340	Wrong Word
11	11	rgba(0, 0, 0, 0.3)	Description text, such as copyright information, that does not require user attention

Icon:

category	colour	Size
Navigation Class	Multiple colors, but no more than three, with the same main color	28px
Menu Action Class	Monochrome, uniform color	22px
Action Tip Class	Relevant to prompt type	30px
Showcase area classification	Icon inherent color	Consistent with follow font size

Responsive design:
Mainly by changing px to rpx, because it does not involve list items at all, and does not consider adaptive layout transformation, only makes elements under different screens present in the same proportion. With iphone-6 as the standard, for iphone-x heterogeneous screens, focus on the operation menu (such as top, bottom, suspension) of the security area, mainly through calc (safe-area-inset-bottom) in CSS.

• Pictures spread across the screen
• Subject text is not required
• PC-side adapter is not considered yet because there are already static sites

3.1.4 Development Specification

• Incrementally, basic functionality is achieved first, then detachment and componentization are considered.
• It can be implemented with simple logic without pulling components apart, using mature libraries without creating components on its own, and configuring or accommodating uses without modifying external libraries to ensure smooth updates.
• For how functionality is implemented, consider the role of the service, weighing computational complexity, network latency, and user perception:

Simple calculations on the applet side

• canvas Draws Posters
• Basic Format Conversion

The server (cloud development) does complex processing, non-real-time computing, or pre-generated content

• markdown to html
• TOC directory
• AI Recognition, Processing
• For read-write databases, try putting write operations in cloud functions.

3.2 Detailed Design

3.2.1 Data Source

Security checks to ensure that cloud functions are triggered in a trustworthy manner:

// View Request Header
if (!req.headers['user-agent'].includes('Coding.net Hook') || 
    !('x-coding-signature' in req.headers) || req.headers['x-coding-signature'].indexOf('sha1=')
    !('x-coding-event' in req.headers) || 'POST' !== req.httpMethod ) {
  return false;
}
// Compute and compare signatures
const theirSignature = req.headers['x-coding-signature'];
const payload = req.body;
const secret = process.env.HOOKTOKEN;
const ourSignature = `sha1=${crypto.createHmac('sha1', secret).update(payload).digest('hex')}`;
return crypto.timingSafeEqual(Buffer.from(theirSignature), Buffer.from(ourSignature));

Each time a commit pushes new code, WebHook pushes the following information (limited to space, omitting unnecessary information)

{
  "ref": "refs/heads/master",
  "commits": [
    {
    "id": "8a175afab1cf117f2e1318f9b7f0bc5d4dd54d45",
    "timestamp": 1592488968000,
    "author": {
      "name": "memakergytcom",
      "email": "me@makergyt.com",
      "username": "memakergytcom"
    },
    "committer": {
      "name": "memakergytcom",
      "email": "me@makergyt.com",
      "username": "memakergytcom"
    },
    "added": [
      "source/_drafts/site.md"
    ],
    "removed": [],
    "modified": [
      "package.json",
      "scripts/fix.js",
      "source/_posts/next.yml",
      "source/_posts/typesetting.md"
    ]}
  ],
  "head_commit":{...},
  "pusher",
  "sender",
  "repository"
}

Keep up to date so focus on head_commit. This information contains changes made by this submission and can be used to synchronize the cloud database based on traversing these changes. However, due to changes that may include non-article files or may not be the target branch, filtering is required:

if ('refs/heads/' + branch === ref) {
  if (filePath.indexOf(dirPrefix) || filePath.slice(-3) !== '.md') {　// Path prefix and article suffix
    continue;
  }
}

To associate a database file with a git repository file, since each commit file does not have unique id information, you can create a connection by filename, using the filename as the slug field (primary key)

let slug = filePath.match(new RegExp(dirPrefix + "([\\s\\S]+)\\.md"))[1];

Since the Push event does not contain file content, requests need to be made through the api

await axios({
  url: `${baseUrl}/${branch}/${filePath}`,
  method: 'get',
  headers: {
    'Authorization': `token ${process.env.CODINGTOKEN}` // Personal Token
  }
});

3.2.2 Data Processing

Extract article information:
Since basic information is required to be written in yaml format at the beginning of markdown, you need to go to json after you get the file contents (String).

const matter = require('hexo-front-matter');
let { title, date, tags, description, categories, _content, cover } = matter.parse(data);

The cover field (cover image) is also not declared and is obtained from the first image of the article

let cover = _content.match(/!\[.*\]\((.+?)\)/);

markdown parses html:
The applet-side environment differs from traditional web pages in that markdown rendering takes place locally and requires a conversion to html first, which is done in advance in the cloud to reduce rendering time:

const md = require('markdown-it')({
  html: true,// Allow rendering of html
}).use(require('markdown-it-footnote'))　// footnote reference

Build Directory
For consistency, chapters are numbered themselves. Directories placed in the sidebar do not resolve to html and need to be processed separately. The markdown-it-anchor plug-in uses the value of the header as the id(markdown-it-anchor), but the id cannot start with a number, cannot contain Chinese and encodeURIComponent (Chinese), but can contain -.

// Insert id for <h>label
id = 'makergyt-' + crypto.createHash('md5').update(title).digest('hex');
// Get a list of all h2-h4 generated directories
const { tocObj } = require('hexo-util');
const data = tocObj(str, { min_depth:2, max_depth: 4 });

3.2.3 Data synchronization

In the widget's documentation, triggering a cloud function can be done through http api (invokeCloudFunction). But invokeCloudFunction requires critical access_ Token, it takes two hours to refresh the fetch, webhook can't be notified in advance. Consider setting up a centralized server to uniformly get and refresh access_ Token, the webhook first makes a request to the central control server and then to the cloud function, but this is obviously not possible because it can only push an address once and has no context. In the meantime, add an intermediate function, then where does the intermediate function go and how to request it? (access_token is also required)

At this point, in Tencent Cloud-Cloud Development Console , found that the cloud function can be triggered directly via Cloud Access HTTP Trigger, so that the address can be used directly as the Url of the WebHook. But business and resource security needs to be addressed [1] Coding's request domain can then be added to the list of WEB security domain names.

Once you get the article information and content, you can synchronize to the appropriate collection of cloud databases, where async/await traversal is used in a loop, and in order to keep the loop until each call is resolved, only for...of Make Asynchronous [2].

for (const file of added) {
  await db.collection('sync_posts').add({
    data
  })
}
for (const file of modified) {
  await db.collection('sync_posts').where({
    slug
  }).update({
    data
  })
}
for (const file of removed) {
  await syncPosts.where({
    slug
  }).remove();
}

3.2.4 Text Rendering

It is almost impossible to leave the original content intact, and the html string rendered with markdown-it does not insert any styles. Direct testing (which provides styles by default based on tags) works as follows:

programme	Effect
rich-text	Code block missing, long content truncated
wxparser	Large spacing, table, code block truncated
towxml	Code block truncated
wemark	No line break between code block and reference section
Parser	Table overflow

Tips: Notice the applet code highlighting and markdown rendering components developed by the Tencent Omi team Comi Is actually used as a template introduction. Consider the subsequent measured effects and contrast rendering speed.

By contrast, component boundaries are overflowed, causing the problem of horizontal scrollbars. In use, there is a flaw that does not support parsing style tags [3]


Parser can solve this problem by controlling the source html style

var document = that.selectComponent("#article").document;
document.getElementsByTagName('table').forEach(tableNode => {
  var div=document.createElement("div");
  div.setStyle("overflow", "scroll");
  div.appendChild(tableNode);
  div._path = tableNode._path;
  tableNode = div;
});

Parser also provides control over the tag style in the source html to influence the rendering, which allows you to change font size, line height, line spacing, and so on, to fit your mobile screen.

//post.wxml
<parser id="article" tag-style="{{tagStyle}}"/>
// post.js
tagStyle: {
  p: 'font-size: 14px;color: #353535;line-height: 2;font-family: "Times New Roman";',
  h2: 'font-size: 18.67px;color: #000;text-align:center;margin: 1em auto;font-weight: 500;font-family: SimHei;',
  h3: 'font-size:16.33px;color: #000;line-height: 2em;font-family: SimHei;',
  h4: 'font-size:14px;color: #000;font-family: SimHei;',
}

For code highlighting, use prism , introduced into the component.

const Prism = require('./prism.js');
...
highlight(content, attrs) {
  content = content.replace(/&lt;/g, '<').replace(/&gt;/g, '>').replace(/quot;/g, '"').replace(/&amp;/g, '&'); // Replace Entity Coding
  attrs["data-content"] = content; // Record the original text for long press copying, etc.
  switch (attrs[lan]) {
    case "javascript":
    case "js":
      return Prism.highlight(content, Prism.languages.javascript, "javascript");
  }
}  

Mathematical formula Latex
There are two main types of latex rendering engines

engine	Characteristic
mathjax	Rich grammar, slow rendering
katex	Less syntax support, fast, only mathml or html output, needed with its CSS and font files

Of course, both are web client rendering, which is not naturally available on the applet side. Consider server-side rendering. The questions are:

• Server-side rendering If external interface is used, encodeUrl is required, but internal rendering is escaped away, rendering is required, replace (//g,'') is invalid
• Server-side rendering If mathjax-node is used and its dependency mathjax version ^2.7.2, all rendering needs to be replaced with rendering, SVG - Unknown character: U+C in MathJax_will often occur Main, MathJax_ Size1, MathJax_ AMS, Matrix Parse Error: Misplaced &
• How to more accurately identify the ATEX of a specific tag in markdown without causing misoperation.

Consider transforming markdown into <img> during the html parsing phase, which is also a more reliable and controllable approach taken by many content platforms. Use here markdown-it-latex2img Plug-in unit

const md = require('markdown-it')({
  html: true,// Enable HTML tags in source
}).use(require('markdown-it-latex2img'),{
    style: "filter: opacity(90%);transform:scale(0.85);text-align:center;" //Optimize display style
  })



3.3 Static Managed

Set up a build plan for the git library to synchronize to the object store after each commit. Use here hexo As a framework for building.

pipeline {
  agent any
  stages {
    stage('detection') {
      steps {
        checkout([
          $class: 'GitSCM',
          branches: [[name: env.GIT_BUILD_REF]],
          userRemoteConfigs: [[url: env.GIT_REPO_URL, credentialsId: env.CREDENTIALS_ID]]
        ])
      }
    }
    stage('structure') {
      steps {
        echo 'Under Construction...'
        sh 'npm install -g cnpm --registry=https://registry.npm.taobao.org'
        sh 'cnpm install'
        sh 'npm run build'
        echo 'Build complete.'
      }
    }
    stage('Parallel phase') {
      parallel {
        stage('Deploy to Tencent Cloud Storage') {
          steps {
            echo 'In Deployment...'
            sh "coscmd config -a $TENCENT_SECRET_ID -s $TENCENT_SECRET_KEY -b $TENCENT_BUCKET -r $TENCENT_REGION"
            sh 'coscmd upload -r public/ /'
            echo 'Deployment Complete'
          }
        }
        stage('Pack') {
          steps {
            sh 'tar -zcf blog.tar.gz public'
            archiveArtifacts(artifacts: 'blog.tar.gz', defaultExcludes: true, fingerprint: true, onlyIfSuccessful: true)
          }
        }
      }
    }
  }
}

Automatically refresh the CDN after building.

// refresh_cdn
const Key = decodeURIComponent(event.Records[0].cos.cosObject.key.replace(/^\/[^/]+\/[^/]+\//,""));
const cdnUrl = `${process.env.CDN_HOST}/${Key}`;
CDN.request('RefreshCdnUrl', {
    'urls.0': cdnUrl
}, (res) => {
  ...
})

4 System Implementation

4.1 Database

Article:

sync_posts = [
  {
    _id: String,
    createTime: String,
    slug: String,
    title: String,
    tags: Array,
    description: String,
    cover: String, // url
    content: String, // html
  }
]
// Security Rules
{
  "read": true, // Public Read
  "write": "get('database.user_info.${auth.openid}').isManager", // Only Administrators can write
}

User Collection

user_favorite = [
  {
    _id:String,
    userId:String,// openid
    postId: String,// Add redundant data to table direct query
    createTime: Date
  }
]
// Security Rules
{
  "read": "doc._openid == auth.openid",// Private Read
  "write": "doc._openid == auth.openid"// Private Writing
}

User Information

user_info = [
  {
    _id: String,
    _openid: String,
    ...userInfo,
    isManager: Boolean,
  }
]
// Security Rules
{
  "read": "doc._openid == auth.openid", // Private Read
  "write": "doc._openid == auth.openid"// Private Writing
}

4.2 Logon

4.2.1 Ordinary Login

With cloud development, you don't need to go through wx.login obtains the login credentials (code) in exchange for user login information, since the caller openid comes with each call to the cloud function. At the same time, some applets bypass user logins because user information can be displayed directly through open-data, whether authorized or not. Some applets save to the database by authorizing user information, which is used in subsequent operations and cannot be updated after the user changes the information. If the user actively deauthorizes through the settings page, but returns with the user's information displayed (showing that they are logged in). This is because the user state information is acquired through onLoad, and the return operation is onShow, so there is a conflict. Users choose to use other nicknames and avatars when they reauthorize their login, and some applets will consider them new users. There are also some applets that require authorization to use regardless of whether user information is required in the business. In fact, the biggest feature of WeChat applet is that it can easily obtain the user identity provided by WeChat and quickly establish the user system within the applet, but none of the above situations properly handle the basic strategy of user login.

Be based on "Come and go freely" The principle can be browsed by tourists, logged in and logged out. Require users to jump to the login page to authorize access to information when it involves functions that require user information to be collected and entered, or user records to be saved. Cloud functions save the user's identity and openid in context to the database, and custom login caches are generated locally in callbacks. Users click Exit to leave it empty.

// cloudfunction/login
const openid = wxContext.OPENID
db.collection('user_info').where({
  _openid: openid
}).get().then(async (res)=> {
  if (res.data.length === 0) {
    db.collection('user_info').add({
      data: {
        _openid: openid,
        ...event.userInfo,
        createTime: db.serverDate(),
      }
    })
  }

The next time you open the applet, you will check the custom login status in the cache to see if the user is logged in. Cloud functions are also called to update user information and usage information (such as opening time, number of opens for subsequent user analysis). No authorization prompt will pop up at the next login, which is a small probability when the user cancels the authorization by himself (or misoperates when wx.openSetting occurs), but once it does, it is a Bug. If you detect users in onShow, you have duplicate logic with normal onLaunch, but you need to detect this behavior. In fact, opening the settings page necessarily leads to onHide, where you can:

// app.js
onHide:function() {
  wx.onAppShow(()=> {
    if(this.globalData.hasLogin) {
      wx.getSetting({
        success: res => {
          if (!res.authSetting['scope.userInfo']) { // Authorization revoked
            this.logout() // Log out directly after returning
          }
        }
      })
    }
    wx.offAppShow();
  })
},

4.2.2 Administrator Authentication

Administrators are article authors, and for administrator identification, consider

• Mobile number: The interface is currently open to non-personal developers and certified applets
• openid: unknown before use, cannot bind in advance
• Other user information, passwords, and so on expose administrative entries

The simplest and most direct data field marker, isMaganer:true, was adopted, which is also used for database security rules.

4.3 Sharing

There are no more than two types of sharing, direct sharing to chat and guided sharing to circles of friends after generating posters. For the former, you need to consider the size of the picture is 5:4, other proportions will result in blanks or cropping. The latter is mainly analyzed here. Drawing an inverted picture through canvas is slow on the applet side, because the content shared by each article is basically fixed, you can consider pre-generation. However, if you share a QR code and a sharer association, you still need to generate it locally. Use components here mini-share . For applets, cloud calls are currently used, which can only be triggered on the applet side.

// Processing parameters
const path = page +'?' + Object.keys(param).map(function (key) {
    return encodeURIComponent(key) + "=" + encodeURIComponent(param[key]);
}).join("&");
// Organization File Name
const fileName = 'wxacode/limitA-' + crypto.createHash('md5').update(path).digest('hex');
// Find the file if a direct return path is found
let getFileRes = await cloud.getTempFileURL({
  fileList: [fileID]
});
// If no regeneration is found
const wxacodeRes = await cloud.openapi.wxacode.get({
  path,
  isHyaline:true
})
// Upload to Cloud Storage
const uploadRes = await cloud.uploadFile({
  cloudPath: fileName + fileSuffix,
  fileContent: wxacodeRes.buffer,
});
// Get Return Temporary Path
getFileRes = await cloud.getTempFileURL({
  fileList: [uploadRes.fileID]
});

There are three ways to generate two-dimensional codes, analysis characteristics

type	Characteristic	Scenarios applicable
A+ C	Limited number, long parameters	Post-build storage is used for long-term effective business and can be used for operations that users like invitation codes can follow for a long time.
B	Infinite number, short parameters	It is not saved after generation, and its scene s are associated with short-term user behavior, such as activity. Live code, which can be converted and reused when associated with a database.

Here due to the article's database_ Class A (wxacode.get) is used because the id defaults to 32 bits, reaches the limit of class B and requires additional information to be associated with it.

4.4 Subscription Messages

For an individual principal, a message can only be sent once after the subscription has been initiated by the applet (to get permission to send). Here, when the user leaves a message, a reply notification is subscribed to, but it cannot be sent to the author (unless the author has a long-term subscription). Since you also need to save to the database, this is done using cloud calls.

// post.js
wx.requestSubscribeMessage({
  tmplIds: [TEMPLATE.REPLY]
})
// cloudfunction/sengMsg
let sendRes = await db.collection('user_msg').add({
  data: {
    _openid: wxContext.OPENID,
    msg:inputMsg,
    createTime:Date.parse(new Date())
  }
});
await cloud.openapi.subscribeMessage.send({
  data: format(data), // Due to the length format limitations of various types of information, processing is required
  touser: wxContext.OPENID,
  templateId: TEMPLATE.REPLY
});

5. Expanded Summary

5.1 Combination Language Sparrow

5.1.1 Sync to Sparrow

• Labels <a name="tqO5w"></a>will be inserted before the title
• Direct copying of the editing interface converts the pictures out of the chain, but direct importing does not.
• Only files (pictures) can be imported locally. External link import is not supported except for third party services joined.
• You can input any type, but output is a unique lake format, and in Update Document When an interface is called, it returns

{
  "status": 400,
  "message": "Sorry, the sparrow is not allowed to pass API modify Lake Format document, please go to sparrow for operation."
}

5.1.2 Synchronization from Sparrow

You can use the good editing experience of the sparrow to write articles and synchronize to other platforms. yuque's webhook sends a webhook.doc_detail gets the content directly. However, the Language Sparrow has made a lot of effective efforts to enrich the content types of documents, and using these features will not guarantee compatibility with other platforms. The slug returned by the delete operation becomes trash-EJA8tL7W, independent of the original slug, and cannot be associated with other platforms through the slug, that is, only the add operation can be synchronized. Therefore, it is impractical and unnecessary to automatically deploy to other platforms for whispering sparrow writing.

5.1.3 workflow

After synchronizing to the sparrow, you can use its rich support types to improve the document content, such as converting text content into more intuitive flowcharts, mind maps, and incorporating demo and code into codepen visual presentations. The data that may be involved can be easily accessed by uploading attachments.
But be aware that:

• Many content platforms often make anti-theft chains for pictures after having a certain user base.
• The current webhook design is insecure, no signature verification is available, and possibly a forged request due to a Webhooks URL leak

5.2 Known issues with applet development

• Actual Initial Animation Carton 500ms
• Native TabBar hiding will jump, animation will black out, all icons will flash when custom TabBar switches, and auto-hiding will show white bars.
• After a few simple round trips to navigate, listeners of event onBeforeUnloadPage_17 have been added, possibly causing memory leak.
• Calling CameraFrameListener. When start starts to listen for frame data, there must be acquisition and processing of pixel data, but this will invalidate all bindtap events in the interface and will not trigger CameraFrameListener by clicking. Stop Stop Function
• Invalid array update operator addToSet in cloud console database management page, after object element passed in Instable Or effective or not

1. Tencent Cloud. CloudBase Document for Cloud Development [EB/OL]. https://cloud.tencent.com/document/product/876/41136. 2020 ↩︎

2. Tory Walker.The-Pitfalls-of-Async-Await-in-Array-Loops[EB/OL].https://medium.com/dailyjs/the-pitfalls-of-async-await-in-array-loops-cf9cf713bfeb. 2020 ↩︎

3. Jinyu Peak. In-depth research and application of rich text capability of applets [EB/OL]. https://developers.weixin.qq.com/community/develop/article/doc/0006e05c1e8dd80b78a8d49f356413. 2019 ↩︎