macOS App automatic distribution App Store exploration and practice

foreword

When it comes to automation, I think many students will think of CI/CD. There is indeed a certain connection between the two, which can be simply understood as the relationship between parent and child collections.

And as the title of the article says, recently I'm researching the automatic distribution of macOS App to the App Store. In layman's terms, I hope to manually build .xcarchive file, export .pkg file and upload App The operation of the Store instead uses shell scripts to automate these steps . Among them, the added Shell script will be added to the appropriate place based on the existing CI/CD implementation.

So, today, this article will also start from the basis of CI, step by step, to take everyone to understand the realization of macOS App automatic distribution App Store.

1 Understand the basics of continuous integration (CI)

Continuous Integration, Continuous Integration, referred to as CI. Here we look at the introduction to CI on Wikipedia :

-- In software engineering, continuous integration (CI) is the practice of merging all developers' working copies to a shared mainline several times a day. Grady Booch first proposed the term CI in his 1991 method, although he did not advocate integrating several times a day. Extreme programming (XP) adopted the concept of CI and did advocate integrating more than once per day – perhaps as many as tens of times per day.

Typically, in our actual development scenarios, CI refers to the practice of integrating a project's build process into a single piece of software. For example, the Shell implementation of the macOS App automated distribution App Store mentioned above will be added to the existing CI/CD process, and its CI process will be as follows:

• A developer (builder) triggers a Jenkins job
• Execute the Job, which will be executed by the Slave Node (build machine) of Jenkins where the Job is located, producing artifacts, such as a .dmg or .pkg file
• Upload products to the product library

Among them, the key is the construction machine (Slave Node), the script implementation of our entire construction process will be executed on the construction machine , such as the implementation of the automatic distribution App Store to be discussed later. The script here is written in Shell, of course, you can also use Google's zx . Interested students can understand it by themselves. It will not be expanded here.

Because, before we know the automatic distribution of macOS App App Store, we need to know what the manual distribution process of macOS App is like, so that we can use automated scripts to implement the steps of manual distribution one by one.

2 Manual distribution (Distribute)

The process of manually distributing macOS App to the App Store is generally done using the GUI interface provided by Xcode . However, the premise of formal operation and use is to have a macOS App that can be published to the App Store and fully configured, which requires you to meet the following 3 points:

• Register as an Apple Developer Program, download Apple Developer in the Apple Store, and then register as a "Valuy 688 Member" in the app's account
• In the background of https://developer.apple.com/ , create certificate related (Bundle Identifier, Provision Profile, Signing Certificate) and register App in Certificates, Identifiers & Profiles and App Store Connect respectively
• Local initialization creates a simple macOS App and associates the previously created Bundle Identifier, Provision Profile, and Signing Certificate

Regarding point 1, I don't think it's hard to understand. Below, we start from creating a macOS App to connect the things to do in points 2 and 3.

2.1 Preliminary preparation (create a complete application)

To create a macOS App project, you can quickly create one through Xcode, open Xcode -> Create a new Xcode Project -> select the Platform (macOS) to create the application -> fill in the project name, Team, Organization Identifier and other information - > Select App Category and App Icons in General, the application I created here is called FEKit.

Use Xcode to open the project's .xcodeproj file, or type in the terminal, in our case:

 open FEKit.xcodeproj

Select the target of the build, here we choose macOS as the protocol (Scheme) and target (Target):

Configure the app's Bundle Identifier, Provision Profile, and Signing Certificate, which can be created by adding an Apple ID in the Apple Developer background or in Xcode's Preferences Shown in:

Among them, if we want to distribute the App Store, we need these 2 certificates:

• Mac App Distribution, used to sign and distribute App Store applications and configure the corresponding Provisioning Profile
• Mac Installer Distribution, for signing app installation packages and submitting to the App Store

Of course, in addition to these two certificates, we also need to create App ID and Provisioning Profile respectively in Identifiers and Profiles on the Certificates, Identifiers & Profiles page, these two steps are relatively simple (not expanded here).

After the certificate, Indentifiers, and Profiles are created, you can download the certificate (.cer) and Provisioning Profile (.provisionprofile) files to the local, and then double-click to open them, and the certificate will be loaded into the keychain corresponding to the computer login. , and the Provisioning Profile will be used by Xcode. Moreover, it is worth mentioning that each certificate is encrypted and requires a matching key to decrypt and use, that is, a certificate (.cer file) corresponds to a key (.p12 file) , and this key is determined by the certificate. Generated by the creator, so you need to select a .certSigningRequest file when creating the certificate:

Then after loading the certificate (.cer file) locally, and ensuring that there is a key (.p12 file) corresponding to the certificate, the certificate in the final login keychain will look like this:

2.2 Xcode manual distribution to the App Store

Next, we can use Xcode's Production -> Archive to build the .xcarchive file:

After the build, Xcode will pop up a window for you to choose Distribute App or Validate App:

Here, we choose Distribution App -> App Store Connect -> Export -> choose Development Team -> Manually manage signing, we will be asked to select the aforementioned Distribution certificate, Installer certificate and Provisioning Profile:

After selecting Next -> Export, you need to select the export file directory, and the FEKit.pkg file will be generated in this directory, and then we can upload the file to App Store Connect through the Transporter tool (or choose Export or Upload in Xcode earlier When selecting Upload), you can view it in TestFlight in the background of App Store Connect :

Therefore, what we usually call uploading to the App Store refers to the TestFlight uploaded to the App Store Connect , and then submit the uploaded files for review in the App Store here, and then proceed to the shelves after the review. Also, it should be noted that the version number of each uploaded .pkg file needs to be one larger than the previous version number (similar to NPM's Package Version).

So, here the whole process of manually distributing the App Store has been introduced. To sum up, there are mainly three steps:

• Build the project to generate the .xcarchive file
• Export .pkg file
• Upload the .pkg file to App Store Connect

Therefore, we need to automate these 3 steps with Shell script , that is, macOS App automatic distribution App Store.

3 Automated distribution (Distribute)

Before introducing the implementation of macOS App automated distribution App Store, let's first understand these three tools:

• xcodebuild is a command line toolkit for Xcode, mainly used to build project related
• altool is a command-line tool built into Xcode for validating and uploading app binaries to the App Store or notarizing apps (Notarize)
• xcrun is also a command-line toolkit for Xcode, mainly used to execute Xcode-related toolchains, such as xrun altool , xrun xcode-select

In the next section explaining the automated distribution of macOS Apps to the App Store, the capabilities provided by these tools will be mentioned separately to complete the previous manual steps. So, let's start with a step-by-step understanding of the automation implementation process, starting with building the .xcarive file.

3.1 Build the .xcarchive file

We can use the xcodebuild.xcarchive command to build the .xcarchive file:

 xcodebuild -archive \
-scheme "FEKit (macOS)" \
-configuration Release \
-archivePath ./Output/FEKit

As you can see, here we use 3 Option, their respective functions:

• -scheme constructed protocol, different target targets usually correspond to different protocols, such as FEKit (iOS) or FEKit (macOS) , the former is IOS, the latter is macOS
• -configuration build configuration, such as Debug or Release, the certificate and signature configuration corresponding to different configurations will be different
• -archivePath Build the directory and file name exported by .xcarchive, which will be exported to the Output directory and named FEKit.xcarchive

Among them, about the existing protocols and configurations of the project, you can use the xcodebuild -list command to view.

3.2 Export .pkg file

After building the .xcarchive file, you need to export the .pkg file according to the modified file, which can also be done using the Option command provided by xcodebuild :

 xcodebuild -exportArchive \
-archivePath ./Output/FEKit.xcarchive \
-exportPath ./Output/Pkgs \
-exportOptionsPlist ./Build/ExportOptions.plist

Among them, about -exportOptionsPlist is the configuration you exported .pkg related, it will be like this:

 <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>destination</key>
    <string>export</string>
    <key>installerSigningCertificate</key>
    <string>你的 Mac AppStore Install 证书 ID</string>
    <key>manageAppVersionAndBuildNumber</key>
    <true/>
    <key>method</key>
    <string>app-store</string>
    <key>provisioningProfiles</key>
    <dict>
        <key>你的 Bundle ID</key>
        <string>你的 Provisioning Profile 的名称</string>
    </dict>
    <key>signingCertificate</key>
    <string>你的证书 ID</string>
    <key>signingStyle</key>
    <string>manual</string>
    <key>teamID</key>
    <string>你的 Team ID</string>
    <key>uploadSymbols</key>
    <true/>
</dict>
</plist>

Of course, if you don't want to manually create or fill in this information, you can use Xcode to manually export the .pkg operation once, and the ExportOptions.plist file will be automatically generated in the exported file directory.

After executing the previous command, the exported .pkg file will be in the -exportPath configured file path, here it is in the /Outputs/Pkgs/ file directory.

3.3 Verify and distribute .pkg files

TODO: Here you can supplement the use of keychain to consume proprietary passwords

Then comes the final step of validating and distributing the .pkg file. This step needs to be done using xcrun and altool . First, execute xcrun altool --validate-app to verify the .pkg file. This is mainly used to confirm whether your application meets the uploading conditions, such as whether to select App Category, App Icon and version number incrementing, etc. The corresponding command is:

 xcrun altool --validate-app \
-f ./Output/Pkgs/FEKit.pkg \
-t macOS \
-u xxxxx \
-p xxxxx \
--show-progress

As you can see, 5 Options are used here, and their respective functions:

• -f The location of the file directory where the .pkg file to be verified is located
• -t Verified target type, such as macOS or IOS
• -u Apple Developer account (Apple ID) for connecting to App Store Connect
• -p App-specific password corresponding to the account (Apple ID)
• --show-progress Used to output the execution of the verification process

Among them, the app-specific password of -p needs to go to the Apple ID background to apply. And, in order to avoid displaying the password in plaintext in the executed command, we can maintain a separate file to store the account and password:

 #!/bin/bash
# App Developer 账号（Apple ID）
user="xxxxxxxx"
# App 专用密码
pwd="xxxxxx"

Correspondingly, different follow-up processing needs to be done according to the result (success or failure) of the xcrun altool --validate-app command execution:

 #!/bin/bash
# app_store_user_pwd.sh 可以单独放到一个隐藏目录，这里只是作为例子所以没有放到隐藏目录
source "./app_store_user_pwd.sh"
echo "Run xcrun altool --validate-app..."
xcrun altool --validate-app --f ./FEKit.pkg -t macOS -u $user -p $pwd --show-progress
if [ $? -eq 0 ]; then
    echo "validate-app success"
    # 执行上传的命令
else
    echo "validate-app fail"
    exit -1;
fi

其中， $?表示上个命令执行结果， 0表示成功， 1表示失败，所以这里if [ $? -eq 0 ]; then命令Whether the execution result is equal to 0 , if it is, it will process the subsequent upload, if not, output the verification failure information and exit.

Then, if the verification passes the .pkg file, we can upload the .pkg file to App Store Connect, which requires executing the xcrun altool --upload-app command:

 xcrun altool --upload-app \
-f ./Output/Pkgs/FEKit.pkg \
-t macOS \
-u xxxxx \
-p xxxxx \
--show-progress

You can see that the uploaded command and the verified command are similar in use, only the first Option is different. So, the whole process of realizing the automatic distribution of the App Store has been introduced here. Since the previous steps are explained in steps, the implementation of the Shell script is separated. Here we combine the Shell implementation mentioned above into one .sh In the file:

 #!/bin/bash
echo "Run xcodebuild archive..."

xcodebuild archive \
-scheme "FEKit (macOS)" \
-configuration Release \
-archivePath ./Output/FEKit

ARCHIVE_FILE=./Output/FEKit.xcarchive
if [ ! -e "$ARCHIVE_FILE" ]; then
    echo ".xarchive doesn't exist";
    exit -1;
fi

echo "Run xcodebuild -exportArchive..."
xcodebuild -exportArchive \
-archivePath ./Output/FEKit.xcarchive \
-exportPath ./Output/Pkgs \
-exportOptionsPlist ./Build/ExportOptions.plist

PKG_FILE=./Output/Pkgs/FEKit.pkg
if [ ! -e "$PKG_FILE" ]; then
    echo ".pkg doesn't exist";
    exit -1;
fi

source "./Build/app_store_user_pwd.sh"
xcrun altool --validate-app --f $PKG_FILE -t macOS -u $user -p $pwd --show-progress
if [ $? -eq 1 ]; then
    echo "altool validate-app fail"
    exit -1;
fi

echo "altool validate-app success"
xcrun altool --upload-app --f $PKG_FILE -t macOS -u $user -p $pwd --show-progress
if [ $? -eq 0 ]; then
    echo "altool --upload-app success"
else
    echo "altool --upload-app fail"
fi

4 Using fastlane to automate distribution

fastlane is a tool that can easily help you complete certificate management, code signing and publishing, and is suitable for iOS, macOS and Android applications. Then, we can also use fastlane to complete the automated distribution of the App Store process using the Shell script above.

First of all, you must install fastlane. The official documentation on this aspect is very detailed and will not be repeated here. When you install fastlane, you can execute fastlane init in the root directory of the application project to initialize its related configuration. During the initialization process, you will be asked to choose the way to use fastlane. Here we can choose manual configuration. , and then it will create a fastlane/Fastfile directory and file in the project root directory, and then when we execute the fastlane xxx command, it will perform specific operations according to the code of the file. The configuration of the generated Fastfile will look like this:

 default_platform(:ios)

platform :ios do
  desc "Description of what the lane does"
  lane :custome_lane do
    # add actions here: https://docs.fastlane.tools/actions
  end
end

Among them, default_platform is used to define a default platform Platform. For example, when we have 2 platforms (iOS and macOS), its configuration needs to be as follows:

 default_platform(:ios)

platform :ios do
  desc "Description of what the lane does"
  lane :custome_lane do
    # add actions here: https://docs.fastlane.tools/actions
  end
end
platform :mac do
  desc "Description of what the lane does"
  lane :custome_lane do
    # add actions here: https://docs.fastlane.tools/actions
  end
end

fastlane custome_lane ，如果我们执行---edadaca210563815b49dc72e32b3c625--- ，由于这里平台默认为---24a10817eedf6b50d6977bcb83b53e4b ios ，所以platorm:ios下的custome_lane ，反之执行fastlane mac custome_lane is platform :mac under custome_lane . Then, for our previous example, only platform:mac is needed:

 default_platform(:ios)

platform :mac do
  desc "Description of what the lane does"
  lane :custome_lane do
    # add actions here: https://docs.fastlane.tools/actions
  end
end

Then, you can write the code we need to automate the distribution of the App Store at platform:mac . The convenience of fastlane is that it implements a lot of out-of-the-box Actions . Here we need to use two Actions, build_mac_app and upload_to_app_store . The former can be used to build and export .pkg files, and the latter can be used to upload .pkg files to Apple Store Connect :

 default_platform(:mac)

platform :mac do
  desc "Description of what the lane does"
  lane :build_upload_appstore do
    # 构建、导出 .pkg 文件
    build_mac_app(
      scheme: "FEKit (macOS)",
      export_method: "app-store",
      output_directory: "./Output/Test",
      output_name: "FEKit",
      export_team_id: "xxxxxx",
      export_options: {
        provisioningProfiles: {
          "com.xxxxxx.xxxx" => "macOSAppStore"
        }
      }
    )
    # 上传 .pkg 到 App Store Connect
    upload_to_app_store(
      pkg: "./Output/Pkgs/FEKit.pkg",
      platform: "osx",
      username: "xxxxxxxxx"
    )
  end
end

Among them, when using upload_to_app_store , it should be noted that here is only the user name of your App Store, and the special password needs to be added to the system environment variable in advance FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD , and then fastlane This environment variable will be read when the upload_to_app_store Action is executed.

Epilogue

Finally, here we use a flowchart to review the entire process of automated distribution of the App Store:

And, I think some students may ask, do we need to understand this as a front-end? I personally think it is necessary, because in some scenarios, such as when doing React Native or Electron development, it is inevitable to come into contact with the concepts of signing, building, notarizing (Notarize) and distributing the App Store of native applications, so, There is still some benefit to doing this by experiencing the process of native applications firsthand (knowing it makes it happen).

If there are inappropriate or wrong expressions in the text, you are welcome to raise an Issue~

like

By reading this article, if you have gained something, you can give a like, this will become the motivation for me to continue to share, thank you~

I am Wuliu. I like to innovate and fiddle with source code. I focus on learning and sharing technology such as source code (Vue 3, Vite), front-end engineering, and cross-end. Welcome to my WeChat public account: Code center .