Skip to main content

11 posts tagged with "Tips & Tricks"

View All Tags

· 9 min read

在写 TiDB Cloud 的 Python SDK 时,我需要实现一个等待集群创建成功的函数 wait_for_available() ,以便用户等待集群变为可用状态。这个函数需要计算集群创建所花费的时间,并在超过用户设置的超时时间时抛出异常。这是一个常见的计算时间差的场景。可以通过记录集群创建的开始时间、获取当前时间,然后计算与开始时间开始时间的差值来实现该功能。伪代码如下:

start_time = GetCurrentTime()
while True:
current_time = GetCurrentTime()
duration = current_time - start_time
if duration > timeout:
raise TimeoutError
if cluster.status == "AVAILABLE":

· 9 min read


When maintaining a Git repository, particularly one for documentation, it is common to have files that haven't been updated in a while. To address this issue, you can link a section or sentence in the document to the corresponding code or an existing issue, so that the document can be updated when the code changes or the issue is resolved. This is useful when starting a new project, but it can be difficult to maintain these links for an existing large project.

To identify potential outdated files, you can use the git log command to retrieve the last commit log of each file and find long-term inactive files. The following sections describe how to generate a last commit report for a repository and how to write the script step by step.

· 11 min read

This document outlines how to utilize code to enhance the accuracy of configuration documentation, particularly for technical writers who might not be well-versed in coding but are responsible for creating or reviewing configuration files. By following the examples in this document, writers can ensure that their configuration files are correct, efficient, and easy to understand, while also reducing the risk of errors or inconsistencies.

· 7 min read


I use webpack to bundle some JS scripts today, see openapi-scripts/pull/16. The following lists details of the project and reasons for bundling:

Step 1. Install webpack-cli

To install webpack-cli, run the following command. For more information about the directory structure, refer to Basic setup.

yarn add webpack webpack-cli --dev

Step 2. Configure a webpack.config.js

For more details about configuration in webpack, see Configuration.

  • Create a webpack.config.js file in the root directory of the repository.

  • Then, configure the webpack.config.js file.


    The entry is ./src/main.js in the repository.

    const path = require('path');

    module.exports = {
    entry: './src/main.js',
    output: {
    filename: 'main.js',
    path: path.resolve(__dirname, 'dist'),
  • Use the preceding configuration file to generate a dist/main.js file.

    npx webpack --config webpack.config.js
  • Oops! An error occurs after executing the preceding command.

    BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
    This is no longer the case. Verify if you need this module and configure a polyfill for it.

    If you want to include a polyfill, you need to:
    - add a fallback 'resolve.fallback: { "http": require.resolve("stream-http") }'
    - install 'stream-http'
    If you don't want to include a polyfill, you can use an empty module like this:
    resolve.fallback: { "http": false }
    @ ./node_modules/@apidevtools/json-schema-ref-parser/lib/options.js 9:21-48
    @ ./node_modules/@apidevtools/json-schema-ref-parser/lib/normalize-args.js 3:16-36
    @ ./node_modules/@apidevtools/json-schema-ref-parser/lib/index.js 6:22-49
    @ ./src/deref.js 2:19-65
    @ ./src/main.js 3:14-35
  • Then, I Google the error message and find the answer in Stack Overflow and Targets in webpack. To use node webpack to compile for usage, add the target: 'node' into the webpack.config.js file.

    const path = require('path');

    module.exports = {
    target: 'node',
    entry: './src/main.js',
    output: {
    filename: 'main.js',
    path: path.resolve(__dirname, 'dist'),
  • Try to execute npx webpack --config webpack.config.js again and a warning message shows:

    $ npx webpack --config webpack.config.js
    asset main.js 484 KiB [emitted] [minimized] (name: main) 1 related asset
    WARNING in configuration
    The 'mode' option has not been set, webpack will fallback to 'production' for this value.
    Set 'mode' option to 'development' or 'production' to enable defaults for each environment.
    You can also set it to 'none' to disable any default behavior. Learn more:

    webpack 5.74.0 compiled with 1 warning in 1990 ms
  • To enables deterministic mangled names for modules and chunks, add mode: 'production' into the webpack.config.js file.

    const path = require('path');

    module.exports = {
    target: 'node',
    entry: './src/main.js',
    output: {
    filename: 'main.js',
    path: path.resolve(__dirname, 'dist'),
    mode: 'production',
  • Try to execute npx webpack --config webpack.config.js again and it works to generate a dist/main.js and dist/main.js.LICENSE.txt.

    $ npx webpack --config webpack.config.js
    json modules 155 KiB
    modules by path ./node_modules/har-schema/lib/ 6.93 KiB 18 modules
    modules by path ./node_modules/ajv/lib/ 5.58 KiB
    ./node_modules/ajv/lib/refs/json-schema-draft-06.json 2.46 KiB
    + 2 modules
    ./node_modules/mime-db/db.json 143 KiB
    webpack 5.74.0 compiled successfully in 2524 ms
  • To test the generated dist/main.js file, run the following command.

    $ node dist/main.js --help
    Usage: postprocess [options] [command]

    Postprocess an OpenAPI document for ReDoc

    -V, --version output the version number
    -h, --help display help for command

    deref <in-filename> [out-filename] Use $RefParser to dereference a JSON schema
    importmd <in-filename> <md-folder> <gen-md> Merge markdown files in <md-folder> to a markdown <gen-file>, and import
    it as $ref to JSON info.description.
    addlogo <in-filename> <url> <alt> <href> Add a logo to JSON info.x-logo
    gencode <in-filename> Generate sample code to JSON as x-code-samples
    devtier <in-filenam> Add sample for creating a dev tier cluster
    help [command] display help for command

Step 3. Modify the Dockerfile

Modify the Dockerfile as follows:

FROM node:16
COPY package.json yarn.lock ./src/ .
RUN yarn install
ENTRYPOINT ["node", "/app/main.js"]
  • The new Dockerfile contains two stages. For more information about multi-stage builds, refer to Use multi-stage builds.

  • The first stage is a build stage, which tells Docker to:

    • build an image starting with the node:16 image and name it build

    • copy all files in the repository to the /build folder


      To ignore some files, you can create a soft link from .gitignore to .dockerignore using ln:

      ln -s .gitignore .dockerignore
    • run yarn install and yarn run build

      The yarn run build command is equivalent to npx webpack --config webpack.config.js, which is the scripts set in package.json. Besides, move dependencies into devDependencies as follows:

      "name": "openapi-scripts",
      "devDependencies": {
      "webpack": "^5.74.0",
      "webpack-cli": "^4.10.0"
      "scripts": {},
      "dependencies": {
      "@apidevtools/json-schema-ref-parser": "^9.0.9",
      "commander": "^9.3.0",
      "openapi-snippet": "^0.14.0"
  • The second stage is a result stage, which builds a image for production.

Step 4. Build the Docker image

To build a image for testing, run the following command to build a test image:

$ docker build -t test .
[1/2] STEP 1/4: FROM node:16 AS build
[1/2] STEP 2/4: WORKDIR /build
--> 73bbc8c7552
[1/2] STEP 3/4: COPY . .
--> 53e13c02e1a
[1/2] STEP 4/4: RUN yarn install && yarn run build
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 22.27s.
yarn run v1.22.19
$ webpack --config webpack.config.js
Done in 6.36s.
--> 5aa7f288822
[2/2] STEP 1/3: FROM node:16
[2/2] STEP 2/3: COPY --from=build /build/dist/main.js ./
--> 0c7bf448572
[2/2] STEP 3/3: ENTRYPOINT ["node", "main.js"]
[2/2] COMMIT test
--> 9dd21695569
Successfully tagged localhost/test:latest

Start a container with the test image and run the ls command to see files in container:

$ docker run -i -t --entrypoint bash 9dd2
root@007:/# ls -la
total 484
dr-xr-xr-x. 1 root root 28 Aug 6 06:43 .
dr-xr-xr-x. 1 root root 28 Aug 6 06:43 ..
drwxr-xr-x. 1 root root 179 Aug 2 01:26 bin
drwxr-xr-x. 2 root root 6 Mar 19 13:44 boot
drwxr-xr-x. 5 root root 360 Aug 6 06:43 dev
drwxr-xr-x. 1 root root 31 Aug 6 06:43 etc
drwxr-xr-x. 1 root root 18 Aug 2 04:21 home
drwxr-xr-x. 1 root root 42 Aug 2 01:26 lib
-rw-r--r--. 1 root root 495164 Aug 7 2022 main.js
drwxr-xr-x. 2 root root 6 Aug 1 00:00 media
drwxr-xr-x. 2 root root 6 Aug 1 00:00 mnt
drwxr-xr-x. 1 root root 27 Aug 2 04:24 opt
dr-xr-xr-x. 161 nobody nogroup 0 Aug 6 06:43 proc
drwx------. 1 root root 20 Aug 2 04:24 root
drwxr-xr-x. 1 root root 42 Aug 6 06:43 run
drwxr-xr-x. 1 root root 20 Aug 2 01:26 sbin
drwxr-xr-x. 2 root root 6 Aug 1 00:00 srv
dr-xr-xr-x. 12 nobody nogroup 0 Aug 5 00:14 sys
drwxrwxrwt. 1 root root 32 Aug 2 04:24 tmp
drwxr-xr-x. 1 root root 19 Aug 1 00:00 usr
drwxr-xr-x. 1 root root 17 Aug 1 00:00 var

It works!

· 4 min read


  1. There is an API definition file open-api-swagger.json in a private repository production/api and I have no write access to it.
  2. I want to deploy a preview website of the open-api-swagger.json:
    • test some Redoc configuration in my repository Oreoxmt/preview-api
    • build the open-api-swagger.json specification file into a HTML file index.html using redoc-cli build
    • deploy it on Netlify or other services

· 6 min read

My Reading List from July 16, 2022 to July 31, 2022.


Reading ListInspirationTags
Things I wish everyone knew about Git (Part I)The opposite of git-push is not git-pull.Git
Things I wish everyone knew about Git (Part II)Good advice is commit early and oftenGit
Shell productivity tips and tricksWe will cover some shell features you can leverage to make your shell do more of the work for you.Tips & Tricks
6 deprecated Linux commands and the tools you should be using insteadSwap your old Linux commands for new and improved alternatives that provide the same functionality, if not more.Tips & Tricks

· 8 min read



· 6 min read

最近在学习 Hadoop 的相关内容,厦门大学数据库实验室提供了相关内容的讲义,但是下载的时候要点击 17 次,文件默认的名称也较为繁琐,改名又需要操作 17 次。所以懒懒的奥利奥同学就开始研究有没有一些不太手动的方法,虽然这个过程花费的时间好像会更多(


应用场景:实现这个页面所有 PPT 的一键下载,以及后续文件名的修改


· 15 min read

Oreo 同学最近开始记笔记和写心情了,但是就是不走寻常路偏偏想整一个自己的网站来写博客。正巧之前有一个 的域名,很快就定下来了 Hexo + GitHub Pages + Cloudflare CDN 的方案,Oreo 自己折腾了一段时间,成功部署了现在你看到的博客并在坚持更新,可喜可贺可喜可贺。

但是有个问题,GitHub Pages 对应的 Repo 存的是渲染过后的 HTML 文件,我们总得有个地方来存原始的 Markdown 文档。于是更新博客首先需要在 _source 目录写 Markdown,然后在根目录执行 hexo generatehexo deploy 更新网站。命令行操作就先不说了,还要研究什么 Node.js、npm 是不是太过分了,在 Windows 配置环境超麻烦的好吗。世界如此美好,我却还要存 node_modules,这样不好,不好。

成,好歹咱也是做过 CI 的,GitHub Actions 也出来一段时间了看起来反响不错,那肯定也可以用 Actions 完成从博客源码到 Pages 部署的完整流程。Google 一搜发现已经有很多很多人做了类似的事情了,但是总有一些小地方不够好看或者是不满足需求,那就只能自己重新造一遍轮子了。