Android build tool-AAPT2 source code analysis (1)

1. What is AAPT2

In the Android development process, we start a build task through the Gradle command, and finally generate the build product "APK" file. The construction process of a regular APK is as follows:

(Quoted from Google official document)

• Compile all resource files, generate resource tables and R files;
• Compile the Java file and package the class file into a dex file;
• Pack resources and dex files to generate unsigned APK files;
• Sign the APK to generate the official package.

Old versions of Android use the AAPT compiler for resource compilation by default. Starting from Android Studio 3.0, AS has enabled AAPT2 as the resource compilation compiler by default. At present, AAPT2 is also the mainstream trend of Android development. Learning the working principle of AAPT2 can help. Android development has a better grasp of the APK construction process, so as to help solve the problems encountered in actual development.

The executable file of AAPT2 is released with the Build Tools of the Android SDK. The AAPT2 tool is included in the build-tools folder of Android Studio, and the directory is (SDK directory/build-tools/version/aapt2).

2. How AAPT2 works

When looking at the Android compilation process, I can't help but think of a question:

Java files need to be compiled to generate class files. I can understand this, but what does the resource file compilation do? Why do we need to compile resources?

With this question in mind, let us study AAPT2 in depth. Unlike AAPT, AAPT2 splits the resource compilation and packaging process into two parts, namely compilation and linking:

compile : compile the resource file into a binary file (flat).

link : Combine the compiled files and pack them into a single file.

By splitting resource compilation into two parts, AAPT2 can improve the performance of resource compilation. For example, if a resource file changes before, AAPT needs to do a full compilation, AAPT2 only needs to recompile the changed file, and then link with other unchanged files.

2.1 Compile command

As described above, the Complie command is used to compile resources, and AAPT2 provides multiple options for use with the Compile command.

The general usage of Complie is as follows:

aapt2 compile path-to-input-files [options] -o output-directory/

After executing the command, AAPT2 will compile the resource file into a .flat format file. The file comparison is as follows.

The Compile command will check the path of the resource file. The path of the input file must conform to the following structure: path/resource-type[-config]/file.

For example, if you save the resource file in the "aapt2" folder and compile it with the Compile command, it will report an error "error: invalid file path'.../aapt2/ic_launcher.png'". Change aapt to "drawable-hdpi" and compile normally.

In Android Studio, you can find the compiled .flat file in the app/build/intermediates/res/merged/ directory. Of course Compile also supports compiling multiple files;

aapt2 compile path-to-input-files1 path-to-input-files2 [options] -o output-directory/

To compile the entire directory, you need to develop a data file. The compiled product is a compressed file that contains all the resources in the directory. The resource directory structure is flattened by the file name.

aapt2 compile --dir .../res [options] -o output-directory/resource.ap_

It can be seen that after compilation, the resource file (png, xml...) will be compiled into a FLAT format file. Drag and drop the FLAT file directly into as to open it, which is garbled. So what exactly is this FLAT file?

2.2 FLAT file

FLAT file is a product file compiled by AAPT2, also called AAPT2 container. The file consists of two parts: file header and resource items:

file header

Resource item

Resource items are divided into two types according to the value of entry_type:

• When the value of entry\_type is equal to 0x00000000, it is of type RES\_TABLE.
• When the value of entry\_type is equal to 0x00000001, it is of type RES\_FILE.

RES_TABLE contains the ResourceTable structure in protobuf format. The data structure is as follows:

// Top level message representing a resource table.
message ResourceTable {
    // 字符串池
    StringPool source_pool = 1;
    // 用于生成资源id
    repeated Package package = 2;
    // 资源叠加层相关
    repeated Overlayable overlayable = 3;
    // 工具版本
    repeated ToolFingerprint tool_fingerprint = 4;
}

The ResourceTable contains:

StringPool : String pool, string constant pool is to reuse the strings in the resource file, thereby reducing the size, the corresponding string in the resource file will be replaced with the index in the string pool.

message StringPool {
  bytes data = 1;
}

Package : contains information about resource id

// 资源id的包id部分，在 [0x00, 0xff] 范围内
message PackageId {
  uint32 id = 1;
}
// 资源id的命名规则
message Package {
  // [0x02, 0x7f) 简单的说，由系统使用
  // 0x7f 应用使用
  // (0x7f, 0xff] 预留Id
  PackageId package_id = 1;
  // 包名
  string package_name = 2;
  // 资源类型，对应string, layout, xml, dimen, attr等，其对应的资源id区间为[0x01, 0xff]
  repeated Type type = 3;
}

The command method of resource id follows the rule of 0xPPTTEEEE, where PP corresponds to PackageId, the resource used by general applications is 7f, TT corresponds to the name of the resource folder, and the last 4 digits are the id of the resource, starting from 0.

The format of RES_FILE type is as follows:

The FLAT file structure of RES_FILE type can refer to the following figure;

As can be seen from the file format shown in the figure above, a FLAT can contain multiple resource items. In the resource item, the header field saves the content of the CompiledFile serialized in protobuf format. In this structure, information such as file name, file path, file configuration and file type are saved. The content of the resource file is saved in the data field. In this way, a file not only saves the external related information of the file, but also contains the original content of the file.

2.3 Compiled source code

Above, we learned the usage of the compile command Compile and the file format of the FLAT file of the compilation product. Next, we learn the compilation process of AAPT2 from the source level by viewing the code, the source code address of this article.

2.3.1 Command execution process

According to common sense, the entry of general functions is related to main. Open Main.cpp, you can find the entry of main function;

int main(int argc, char** argv) {
#ifdef _WIN32
    ......
    //参数格式转换
    argv = utf8_argv.get();
#endif
    //具体的实现MainImpl中
    return MainImpl(argc, argv);
}

In MainImpl, first get the parameter part from the input, and then create a MainCommand to execute the command.

int MainImpl(int argc, char** argv) {
    if (argc < 1) {
        return -1;
    }
    // 从下标1开始的输入，保存在args中
    std::vector<StringPiece> args;
    for (int i = 1; i < argc; i++) {
        args.push_back(argv[i]);
    }
    //省略部分代码，这部分代码用于打印信息和错误处理
    //创建一个MainCommand
    aapt::MainCommand main_command(&printer, &diagnostics);
    // aapt2的守护进程模式，
    main_command.AddOptionalSubcommand( aapt::util::make_unique<aapt::DaemonCommand>(&fout, &diagnostics));
    // 调用Execute方法执行命令
    return main_command.Execute(args, &std::cerr);
}

MainCommand inherits from Command. In the MainCommand initialization method, multiple secondary commands will be added. Through the class name, it can be easily inferred that these Commands correspond to the secondary commands viewed by the terminal through commands.

explicit MainCommand(text::Printer* printer, IDiagnostics* diagnostics)
 : Command("aapt2"), diagnostics_(diagnostics) {
    //对应Compile 命令
    AddOptionalSubcommand(util::make_unique<CompileCommand>(diagnostics));
    //对应link 命令
    AddOptionalSubcommand(util::make_unique<LinkCommand>(diagnostics));
    AddOptionalSubcommand(util::make_unique<DumpCommand>(printer, diagnostics));
    AddOptionalSubcommand(util::make_unique<DiffCommand>());
    AddOptionalSubcommand(util::make_unique<OptimizeCommand>());
    AddOptionalSubcommand(util::make_unique<ConvertCommand>());
    AddOptionalSubcommand(util::make_unique<VersionCommand>());
}

The AddOptionalSubcommand method is defined in the base class Command, the content is relatively simple, and the incoming subCommand is stored in an array.

void Command::AddOptionalSubcommand(std::unique_ptr<Command>&& subcommand, bool experimental) {
    subcommand->full_subcommand_name_ = StringPrintf("%s %s", name_.data(), subcommand->name_.data());
    if (experimental) {
        experimental_subcommands_.push_back(std::move(subcommand));
    } else {
    subcommands_.push_back(std::move(subcommand));
    }
}

Next, let's main_command.Execute 1617f5f45e4e9b. From the method name, it can be inferred that there is code related to instruction execution in this method. There is no implementation of the Execute method in MainCommand, it should be implemented in the parent class, and then search in the Command class, and it really is here.

int Command::Execute(const std::vector<StringPiece>& args, std::ostream* out_error) {
    TRACE_NAME_ARGS("Command::Execute", args);
    std::vector<std::string> file_args;
    for (size_t i = 0; i < args.size(); i++) {
        const StringPiece& arg = args[i];
        // 参数不是 '-'
        if (*(arg.data()) != '-') {
        //是第一个参数
        if (i == 0) {
            for (auto& subcommand : subcommands_) {
                //判断是否是子命令
                if (arg == subcommand->name_ || (!subcommand->short_name_.empty() && arg == subcommand->short_name_)) {
                //执行子命令的Execute 方法，传入参数向后移动一位
                return subcommand->Execute( std::vector<StringPiece>(args.begin() + 1, args.end()), out_error);
            }
        }
        //省略部分代码
    //调用Action方法，在执行二级命令时，file_args保存的是位移后的参数
    return Action(file_args);
}

In the Execute method, the parameter will be judged first. If the first parameter of the parameter hits the second-level command (Compile, link,...), the Execute method of the second-level command will be called. Referring to the example of the compilation command above, it can be seen that under normal circumstances, the judgment of the second-level command will be hit here, and the Execute method of the second-level command will be called.

In the same level directory of Command.cpp, you can find Compile.cpp, whose Execute inherits from the parent class. But because the parameters have been shifted, the Action method will eventually be executed. The Action method can be found in Compile.cpp, and in the implementation classes of other secondary commands (Link.cpp, Dump.cpp...), the core processing processing also has the Action method. The schematic diagram of the overall call is as follows:

Before starting to look at the Action code, let's take a look at the content of Compile.cpp's header file Compile.h. When CompileCommand is initialized, both mandatory and optional parameters will be initialized and defined.

SetDescription("Compiles resources to be linked into an apk.");
AddRequiredFlag("-o", "Output path", &options_.output_path, Command::kPath);
AddOptionalFlag("--dir", "Directory to scan for resources", &options_.res_dir, Command::kPath);
AddOptionalFlag("--zip", "Zip file containing the res directory to scan for resources", &options_.res_zip, Command::kPath);
AddOptionalFlag("--output-text-symbols", "Generates a text file containing the resource symbols in the\n" "specified file", &options_.generate_text_symbols_path, Command::kPath);
AddOptionalSwitch("--pseudo-localize", "Generate resources for pseudo-locales " "(en-XA and ar-XB)", &options_.pseudolocalize);
AddOptionalSwitch("--no-crunch", "Disables PNG processing", &options_.no_png_crunch);
AddOptionalSwitch("--legacy", "Treat errors that used to be valid in AAPT as warnings", &options_.legacy_mode);
AddOptionalSwitch("--preserve-visibility-of-styleables", "If specified, apply the same visibility rules for\n" "styleables as are used for all other resources.\n" "Otherwise, all stylesables will be made public.", &options_.preserve_visibility_of_styleables);
AddOptionalFlag("--visibility", "Sets the visibility of the compiled resources to the specified\n" "level. Accepted levels: public, private, default", &visibility_);
AddOptionalSwitch("-v", "Enables verbose logging", &options_.verbose);
AddOptionalFlag("--trace-folder", "Generate systrace json trace fragment to specified folder.", &trace_folder_);

The compilation options listed on the official website are not complete. After using compile -h to print the information, you will find that the printed information is consistent with the settings in the code.

The execution flow of the Action method can be summarized as:

1) The resource type will be judged based on the input parameters, and the corresponding file loader (file_collection) will be created.

2) Determine the type of output file according to the input output path, and create a corresponding archiver (archive\_writer). Archive\_writer is passed down in the subsequent call chain, and finally the compiled file is written to by archive_writer The output directory.

3) Call the Compile method to execute compilation.

The file read and write objects involved in processes 1 and 2 are as follows.

The simplified main process code is as follows:

int CompileCommand::Action(const std::vector<std::string>& args) {
    //省略部分代码....
    std::unique_ptr<io::IFileCollection> file_collection;
    //加载输入资源，简化逻辑，下面会省略掉校验的代码
    if (options_.res_dir && options_.res_zip) {
        context.GetDiagnostics()->Error(DiagMessage() << "only one of --dir and --zip can be specified");
        return 1;
    } else if (options_.res_dir) {
        //加载目录下的资源文件...
        file_collection = io::FileCollection::Create(options_.res_dir.value(), &err);
        //...
    }else if (options_.res_zip) {
        //加载压缩包格式的资源文件...
        file_collection = io::ZipFileCollection::Create(options_.res_zip.value(), &err);
        //...
    } else {
        //也是FileCollection，先定义collection，通过循环依次添加输入文件，再拷贝到file_collection
        file_collection = std::move(collection);
    }
    std::unique_ptr<IArchiveWriter> archive_writer;
    //产物输出文件类型
    file::FileType output_file_type = file::GetFileType(options_.output_path);
    if (output_file_type == file::FileType::kDirectory) {
        //输出到文件目录
        archive_writer = CreateDirectoryArchiveWriter(context.GetDiagnostics(), options_.output_path);
    } else {
        //输出到压缩包
        archive_writer = CreateZipFileArchiveWriter(context.GetDiagnostics(), options_.output_path);
    }
    if (!archive_writer) {
        return 1;
    }
    return Compile(&context, file_collection.get(), archive_writer.get(), options_);
}

In the Compile method, the input resource file name will be compiled. The processing method of each resource file is as follows:

• Parse the input resource path to obtain resource name, extension and other information;
• Judge the file type according to path, and then set different compilation functions for compile_func;
• The file name of the generated output. The output is the FLAT file name, which will splice the entire path, and finally generate the similar file name in the above case—"drawable-hdpi\_ic\_launcher.png.flat";
• Pass in various parameters and call the compile_func method to execute compilation.

ResourcePathData contains information such as resource path, resource name, resource extension, etc. AAPT2 will obtain the type of resource from it.

 int Compile(IAaptContext* context, io::IFileCollection* inputs, IArchiveWriter* output_writer, CompileOptions& options) {
    TRACE_CALL();
    bool error = false;
    // 编译输入的资源文件
    auto file_iterator  = inputs->Iterator();
    while (file_iterator->HasNext()) {
        // 省略部分代码（文件校验相关...）
        std::string err_str;
        ResourcePathData path_data;
        // 获取path全名，用于后续文件类型判断
        if (auto maybe_path_data = ExtractResourcePathData(path, inputs->GetDirSeparator(), &err_str)) {
            path_data = maybe_path_data.value();
        } else {
            context->GetDiagnostics()->Error(DiagMessage(file->GetSource()) << err_str);
            error = true;
            continue;
        }
 
        // 根据文件类型，选择编译方法，这里的 CompileFile 是函数指针，指向一个编译方法。
        // 使用使用设置为CompileFile方法
        auto compile_func = &CompileFile;
        // 如果是values目录下的xml资源，使用 CompileTable 方法编译，并修改扩展名为arsc
        if (path_data.resource_dir == "values" && path_data.extension == "xml") {
            compile_func = &CompileTable;
            // We use a different extension (not necessary anymore, but avoids altering the existing // build system logic).
            path_data.extension = "arsc";
        } else if (const ResourceType* type = ParseResourceType(path_data.resource_dir)) {
            // 解析资源类型，如果kRaw类型，执行默认的编译方法，否则执行如下代码。
            if (*type != ResourceType::kRaw) {
                //xml路径或者文件扩展为.xml
                if (*type == ResourceType::kXml || path_data.extension == "xml") {
                    // xml类，使用CompileXml方法编译
                    compile_func = &CompileXml;
                } else if ((!options.no_png_crunch && path_data.extension == "png") || path_data.extension == "9.png") { //如果后缀名是.png并且开启png优化或者是点9图类型
                    // png类，使用CompilePng方法编译
                    compile_func = &CompilePng;
                }
            }
        } else {
            // 不合法的类型，输出错误信息，继续循环
            context->GetDiagnostics()->Error(DiagMessage() << "invalid file path '" << path_data.source << "'");
            error = true;
            continue;
        } 
        // 校验文件名中是否有.
        if (compile_func != &CompileFile && !options.legacy_mode && std::count(path_data.name.begin(), path_data.name.end(), '.') != 0) {
            error = true;
            context->GetDiagnostics()->Error(DiagMessage(file->GetSource()) << "file name cannot contain '.' other than for" << " specifying the extension");
            continue;
        }
        // 生成产物文件名，这个方法会生成完成的flat文件名，例如上文demo中的 drawable-hdpi_ic_launcher.png.flat
        const std::string out_path = BuildIntermediateContainerFilename(path_data);
        // 执行编译方法
        if (!compile_func(context, options, path_data, file, output_writer, out_path)) {
            context->GetDiagnostics()->Error(DiagMessage(file->GetSource()) << "file failed to compile"); error = true;
        }
    }
    return error ? 1 : 0;
}

There are four compilation functions for different resource types:

• CompileFile
• CompileTable
• CompileXml
• CompilePng

CompileXml will not be executed for the XML files in the raw directory. It is guessed that the resources under raw are copied directly to the APK and will not be compiled with XML optimization. The resources in the values directory will not only perform CompileTable compilation, but also modify the extension of the resource file. It can be considered that in addition to CompileFile, other compilation methods will more or less process the original resources and write the compiled FLAT file. The process of this part is shown in the figure below:

The main process of compiling command execution ends here. Through source code analysis, we can know that AAPT2 compiles the input file into a FLAT file. Below, we are further analyzing 4 compilation methods.

2.3.2 Four compilation functions

CompileFile

The function first constructs the ResourceFile object and the original file data, and then calls WriteHeaderAndDataToWriter to write the data to the output file (flat).

static bool CompileFile(IAaptContext* context, const CompileOptions& options, const ResourcePathData& path_data, io::IFile* file, IArchiveWriter* writer, const std::string& output_path) {
    TRACE_CALL();
    if (context->IsVerbose()) {
        context->GetDiagnostics()->Note(DiagMessage(path_data.source) << "compiling file");
    }
    // 定义ResourceFile 对象，保存config，source等信息
    ResourceFile res_file;
    res_file.name = ResourceName({}, *ParseResourceType(path_data.resource_dir), path_data.name);
    res_file.config = path_data.config;
    res_file.source = path_data.source;
    res_file.type = ResourceFile::Type::kUnknown; //这类型下可能有xml，png或者其他的什么，统一设置类型为unknow。
    // 原始文件数据
    auto data = file->OpenAsData();
    if (!data) {
        context->GetDiagnostics()->Error(DiagMessage(path_data.source) << "failed to open file ");
        return false;
    }
    return WriteHeaderAndDataToWriter(output_path, res_file, data.get(), writer, context->GetDiagnostics());
}

The content of ResourceFile is relatively simple, and the WriteHeaderAndDataToWriter method will be called after completing the assignment of the file-related information.

In the WriteHeaderAndDataToWriter method, the previously created archive_writer (searchable in this article, this archive will be passed down after the creation is completed) is packaged once, and the packaged ContainerWriter has the ability to write ordinary files and serialize in protobuf format. .

pb provides the ZeroCopyStream interface user data read and write and serialization/deserialization operations.

The process of WriteHeaderAndDataToWriter can be simply summarized as:

• IArchiveWriter.StartEntry, open the file and prepare for writing;
• ContainerWriter.AddResFileEntry, write data;
• IArchiveWriter.FinishEntry, close the file and release the memory.

static bool WriteHeaderAndDataToWriter(const StringPiece& output_path, const ResourceFile& file, io::KnownSizeInputStream* in, IArchiveWriter* writer, IDiagnostics* diag) {
    // 打开文件
    if (!writer->StartEntry(output_path, 0)) {
        diag->Error(DiagMessage(output_path) << "failed to open file");
        return false;
    }
    // Make sure CopyingOutputStreamAdaptor is deleted before we call writer->FinishEntry().
    {
        // 对write做一层包装，用来写protobuf数据
        CopyingOutputStreamAdaptor copying_adaptor(writer);
        ContainerWriter container_writer(©ing_adaptor, 1u);
        //把file按照protobuf格式序列化，序列化后的文件是 pb_compiled_file，这里的file文件是ResourceFile文件，包含了原始文件的路径，配置等信息
        pb::internal::CompiledFile pb_compiled_file;
        SerializeCompiledFileToPb(file, &pb_compiled_file);
        // 再把pb_compiled_file 和 in（原始文件） 写入到产物文件中
        if (!container_writer.AddResFileEntry(pb_compiled_file, in)) {
            diag->Error(DiagMessage(output_path) << "failed to write entry data");
            return false;
        }
    }
    // 退出写状态
    if (!writer->FinishEntry()) {
        diag->Error(DiagMessage(output_path) << "failed to finish writing data");
        return false;
    }
    return true;
}

Let's look at these three methods separately. First, StartEntry and FinishEntry. This method is in Archive.cpp . The implementation of ZipFileWriter and DirectoryWriter is somewhat different, but the logic is the same. Here we only analyze the implementation of DirectoryWriter.

StartEntry, call fopen to open the file.

bool StartEntry(const StringPiece& path, uint32_t flags) override {
    if (file_) {
        return false;
    }
    std::string full_path = dir_;
    file::AppendPath(&full_path, path);
    file::mkdirs(file::GetStem(full_path).to_string());
    //打开文件
    file_ = {::android::base::utf8::fopen(full_path.c_str(), "wb"), fclose};
    if (!file_) {
        error_ = SystemErrorCodeToString(errno);
        return false;
    }
    return true;
}

FinishEntry , call reset to release the memory.

bool FinishEntry() override {
    if (!file_) {
        return false;
    }
    file_.reset(nullptr);
    return true;
}

ContainerWriter class is defined in the Container.cpp class file. In the construction method of the ContainerWriter class, you can find the writing code of the file header, and its format is the same as that described in the "FLAT format" section above.

// 在类的构造方法中，写入文件头的信息
ContainerWriter::ContainerWriter(ZeroCopyOutputStream* out, size_t entry_count)
  : out_(out), total_entry_count_(entry_count), current_entry_count_(0u) {
    CodedOutputStream coded_out(out_);
    // 魔法数据，kContainerFormatMagic = 0x54504141u
    coded_out.WriteLittleEndian32(kContainerFormatMagic);  
    // 版本号，kContainerFormatVersion = 1u
    coded_out.WriteLittleEndian32(kContainerFormatVersion);
    // 容器中包含的条目数 total_entry_count_是在ContainerReader构造时赋值，值由外部传入
    coded_out.WriteLittleEndian32(static_cast<uint32_t>(total_entry_count_));
    if (coded_out.HadError()) {
        error_ = "failed writing container format header";
    }
}

Call the AddResFileEntry method of ContainerWriter to write the content of the resource item.

// file:protobuf格式的信息文件，in：原始文件
bool ContainerWriter::AddResFileEntry(const pb::internal::CompiledFile& file, io::KnownSizeInputStream* in) {
    // 判断条目数量，大于设定数量就直接报错
    if (current_entry_count_ >= total_entry_count_) {
        error_ = "too many entries being serialized";
        return false;
    }
    // 条目++
    current_entry_count_++;
    constexpr const static int kResFileEntryHeaderSize = 12; 、
    //输出流
    CodedOutputStream coded_out(out_);
    //写入资源类型
    coded_out.WriteLittleEndian32(kResFile);
 
    const ::google::protobuf::uint32
    // ResourceFile 文件长度 ，该部分包含了当前文件的路径，类型，配置等信息
    header_size = file.ByteSize();
    const int header_padding = CalculatePaddingForAlignment(header_size);
    // 原始文件长度
    const ::google::protobuf::uint64 data_size = in->TotalSize();
    const int data_padding = CalculatePaddingForAlignment(data_size);
    // 写入数据长度，计算公式：kResFileEntryHeaderSize（固定12） + ResourceFile文件长度 + header_padding + 原始文件长度 + data_padding
    coded_out.WriteLittleEndian64(kResFileEntryHeaderSize + header_size + header_padding + data_size + data_padding);
 
    // 写入文件头长度
    coded_out.WriteLittleEndian32(header_size);
    // 写入数据长度
    coded_out.WriteLittleEndian64(data_size);
    // 写入“头信息”
    file.SerializeToCodedStream(&coded_out);
    // 对齐
    WritePadding(header_padding, &coded_out);
    // 使用Copy之前需要调用Trim（至于为什么，其实也不太清楚，好在我们学习AAPT2，了解底层API的功能即可。如果有读者知道，希望赐教）
    coded_out.Trim();
    // 异常判断
    if (coded_out.HadError()) {
        error_ = "failed writing to output"; return false;
    } if (!io::Copy(out_, in)) {  //资源数据（源码中也叫payload，可能是png，xml，或者XmlNode）
        if (in->HadError()) {
            std::ostringstream error;
            error << "failed reading from input: " << in->GetError();
            error_ = error.str();
        } else {
            error_ = "failed writing to output";
        }
        return false;
    }
    // 对其
    WritePadding(data_padding, &coded_out);
    if (coded_out.HadError()) {
        error_ = "failed writing to output";
        return false;
    }
    return true;
}

In this way, the FLAT file is written, and the product file not only contains the resource content, but also contains the file name, path, configuration and other information.

CompilePng

This method is similar to the CompileFile process. The difference is that the PNG image will be processed first (png optimization and 9-image processing), and after the processing is completed, the FLAT file will be written.

​static bool CompilePng(IAaptContext* context, const CompileOptions& options, const ResourcePathData& path_data, io::IFile* file, IArchiveWriter* writer, const std::string& output_path) {
    //..省略部分校验代码
    BigBuffer buffer(4096);
   // 基本一样的代码，区别是type不一样
    ResourceFile res_file;
    res_file.name = ResourceName({}, *ParseResourceType(path_data.resource_dir), path_data.name);
    res_file.config = path_data.config;
    res_file.source = path_data.source;
    res_file.type = ResourceFile::Type::kPng;
 
    {
        // 读取资源内容到data中
        auto data = file->OpenAsData();
        // 读取结果校验
        if (!data) {
            context->GetDiagnostics()->Error(DiagMessage(path_data.source) << "failed to open file ");
            return false;
        }
        // 用来保存输出流
        BigBuffer crunched_png_buffer(4096);
        io::BigBufferOutputStream crunched_png_buffer_out(&crunched_png_buffer);
 
        // 对PNG图片做优化
        const StringPiece content(reinterpret_cast<const char*>(data->data()), data->size());
        PngChunkFilter png_chunk_filter(content);
        std::unique_ptr<Image> image = ReadPng(context, path_data.source, &png_chunk_filter);
        if (!image) {
            return false;
        }
         
        // 处理.9图
        std::unique_ptr<NinePatch> nine_patch;
        if (path_data.extension == "9.png") {
            std::string err;
            nine_patch = NinePatch::Create(image->rows.get(), image->width, image->height, &err);
            if (!nine_patch) {
                context->GetDiagnostics()->Error(DiagMessage() << err); return false;
            }
            // 移除1像素的边框
            image->width -= 2;
            image->height -= 2;
            memmove(image->rows.get(), image->rows.get() + 1, image->height * sizeof(uint8_t**));
            for (int32_t h = 0; h < image->height; h++) {
                memmove(image->rows[h], image->rows[h] + 4, image->width * 4);
            } if (context->IsVerbose()) {
                context->GetDiagnostics()->Note(DiagMessage(path_data.source) << "9-patch: " << *nine_patch);
            }
        }
 
        // 保存处理后的png到 &crunched_png_buffer_out
        if (!WritePng(context, image.get(), nine_patch.get(), &crunched_png_buffer_out, {})) {
            return false;
        }
 
        // ...省略部分图片校验代码,这部分代码会比较优化后的图片和原图片的大小，如果优化后比原图片大，则使用原图片。（PNG优化后是有可能比原图片还大的）
    }
    io::BigBufferInputStream buffer_in(&buffer);
    // 和 CompileFile 调用相同的方法，写入flat文件，资源文件内容是
    return WriteHeaderAndDataToWriter(output_path, res_file, &buffer_in, writer, context->GetDiagnostics());
}

AAPT2's compression of PNG images can be divided into three aspects:

• Can RGB be converted into grayscale;
• Whether the transparent channel can be deleted;
• Is there a maximum of 256 colors (Indexed_color optimization).

PNG optimization , students who are interested can take a look

After the PNG processing is completed, the WriteHeaderAndDataToWriter will also be called to write the data. This part of the content can be read for the analysis above and will not be repeated.

CompileXml

This method first parses the XML, and then creates an XmlResource, which contains information such as the resource name, configuration, and type. Write the output file through the FlattenXmlToOutStream function.

static bool CompileXml(IAaptContext* context, const CompileOptions& options,
                       const ResourcePathData& path_data, io::IFile* file, IArchiveWriter* writer,
                       const std::string& output_path) {
  // ...省略校验代码
  std::unique_ptr<xml::XmlResource> xmlres;
  {
    // 打开xml文件
    auto fin = file->OpenInputStream();
    // ...省略校验代码
    // 解析XML
    xmlres = xml::Inflate(fin.get(), context->GetDiagnostics(), path_data.source);
    if (!xmlres) {
      return false;
    }
  }
  //
  xmlres->file.name = ResourceName({}, *ParseResourceType(path_data.resource_dir), path_data.name);
  xmlres->file.config = path_data.config;
  xmlres->file.source = path_data.source;
  xmlres->file.type = ResourceFile::Type::kProtoXml;
 
  // 判断id类型的资源是否有id合法（是否有id异常，如果有提示“has an invalid entry name”）
  XmlIdCollector collector;
  if (!collector.Consume(context, xmlres.get())) {
    return false;
  }
 
  // 处理aapt:attr内嵌资源
  InlineXmlFormatParser inline_xml_format_parser;
  if (!inline_xml_format_parser.Consume(context, xmlres.get())) {
    return false;
  }
 
  // 打开输出文件
  if (!writer->StartEntry(output_path, 0)) {
    context->GetDiagnostics()->Error(DiagMessage(output_path) << "failed to open file");
    return false;
  }
 
  std::vector<std::unique_ptr<xml::XmlResource>>& inline_documents =
      inline_xml_format_parser.GetExtractedInlineXmlDocuments();
 
  {
    // 和CompileFile 类似，创建可处理protobuf格式的writer，用于protobuf格式序列化
    CopyingOutputStreamAdaptor copying_adaptor(writer);
    ContainerWriter container_writer(©ing_adaptor, 1u + inline_documents.size());
 
    if (!FlattenXmlToOutStream(output_path, *xmlres, &container_writer,
                               context->GetDiagnostics())) {
      return false;
    }
    // 处理内嵌的元素（aapt:attr）
    for (const std::unique_ptr<xml::XmlResource>& inline_xml_doc : inline_documents) {
      if (!FlattenXmlToOutStream(output_path, *inline_xml_doc, &container_writer,
                                 context->GetDiagnostics())) {
        return false;
      }
    }
  }
  // 释放内存
  if (!writer->FinishEntry()) {
    context->GetDiagnostics()->Error(DiagMessage(output_path) << "failed to finish writing data");
    return false;
  }
  // 编译选项部分，省略
  return true;
}

In the method of compiling XML, ResourceFile is not created like the previous two methods, but XmlResource is created to save the relevant information of the XML resource. Its structure contains the following content:

After executing the Inflate method, XmlResource will contain resource information and XML dom tree information. InlineXmlFormatParser is used to parse out the inline attributes aapt:attr .

Using AAPT's embedded resource format, all multiple resources can be defined in the same XML file. If resource reuse is not required, this method is more compact. The XML tag tells AAPT that the child tags of the tag should be treated as resources and extracted into its own resource file. The value in the attribute name is used to specify where the embedded resource is used within the parent tag. AAPT will generate resource files and names for all embedded resources. Applications built using this embedded format are compatible with all versions of Android. ——Official documents

The parsed FlattenXmlToOutStream will first call the SerializeCompiledFileToPb method to convert the relevant information of the resource file into protobuf format, and then call SerializeXmlToPb to convert the previously parsed Element node information into XmlNode (protobuf structure, also defined in Resources ), and then Convert the generated XmlNode into a string. Finally, add it to the resource item of the FLAT file through the AddResFileEntry method above. It can be seen here that the FLAT file file generated by XML can contain multiple resource items in one FLAT file.

static bool FlattenXmlToOutStream(const StringPiece& output_path, const xml::XmlResource& xmlres,
                                  ContainerWriter* container_writer, IDiagnostics* diag) {
  // 序列化CompiledFile部分
  pb::internal::CompiledFile pb_compiled_file;
  SerializeCompiledFileToPb(xmlres.file, &pb_compiled_file);
 
  // 序列化XmlNode部分
  pb::XmlNode pb_xml_node;
  SerializeXmlToPb(*xmlres.root, &pb_xml_node);
 
  // 专程string格式的流，这里可以再找源码看看
  std::string serialized_xml = pb_xml_node.SerializeAsString();
  io::StringInputStream serialized_in(serialized_xml);
  // 保存到资源项中
  if (!container_writer->AddResFileEntry(pb_compiled_file, &serialized_in)) {
    diag->Error(DiagMessage(output_path) << "failed to write entry data");
    return false;
  }
  return true;
}

The protobuf format processing method (SerializeXmlToPb) is in ProtoSerialize.cpp, which realizes the replication of the node structure through traversal and recursion. Interested readers can view the source code.

CompileTable

The CompileTable function is used to process the resources under values. As you can see from the above, the resources under values will be modified and expanded to arsc at compile time. The final output file name is *.arsc.flat, and the effect is as follows:

At the beginning of the function, the resource file will be read, xml parsing is completed and saved as a ResourceTable structure, and then it is converted to pb::ResourceTable in protobuf format through SerializeTableToPb, and then SerializeWithCachedSizes is called to serialize the table in protobuf format to the output file.

static bool CompileTable(IAaptContext* context, const CompileOptions& options,
                         const ResourcePathData& path_data, io::IFile* file, IArchiveWriter* writer,
                         const std::string& output_path) {
  // Filenames starting with "donottranslate" are not localizable
  bool translatable_file = path_data.name.find("donottranslate") != 0;
  ResourceTable table;
  {
    // 读取文件
    auto fin = file->OpenInputStream();
    if (fin->HadError()) {
      context->GetDiagnostics()->Error(DiagMessage(path_data.source)
          << "failed to open file: " << fin->GetError());
      return false;
    }
 
    // 创建XmlPullParser，设置很多handler，用于xml解析
    xml::XmlPullParser xml_parser(fin.get());
     
    // 设置解析选项
    ResourceParserOptions parser_options;
    parser_options.error_on_positional_arguments = !options.legacy_mode;
    parser_options.preserve_visibility_of_styleables = options.preserve_visibility_of_styleables;
    parser_options.translatable = translatable_file;
    parser_options.visibility = options.visibility;
     
    // 创建ResourceParser，并把结果保存到ResourceTable中
    ResourceParser res_parser(context->GetDiagnostics(), &table, path_data.source, path_data.config,
        parser_options);
    // 执行解析
    if (!res_parser.Parse(&xml_parser)) {
      return false;
    }
  }
  // 省略部分校验代码
 
  // 打开输出文件
  if (!writer->StartEntry(output_path, 0)) {
    context->GetDiagnostics()->Error(DiagMessage(output_path) << "failed to open");
    return false;
  }
 
  {
    // 和前面一样，创建ContainerWriter 用于写文件
    CopyingOutputStreamAdaptor copying_adaptor(writer);
    ContainerWriter container_writer(©ing_adaptor, 1u);
 
    pb::ResourceTable pb_table;
    // 把ResourceTable序列化为pb::ResourceTable
    SerializeTableToPb(table, &pb_table, context->GetDiagnostics());
    // 写入数据项pb::ResourceTable
    if (!container_writer.AddResTableEntry(pb_table)) {
      context->GetDiagnostics()->Error(DiagMessage(output_path) << "failed to write");
      return false;
    }
  }
 
  if (!writer->FinishEntry()) {
    context->GetDiagnostics()->Error(DiagMessage(output_path) << "failed to finish entry");
    return false;
  }
  // ...省略部分代码...
  }
 
  return true;
}

3. Questions and summary

Through the above study, we know that AAPT2 is a build tool for Android resource packaging. It divides resource compilation into two parts: compilation and linking. Among them, compilation is to uniformly compile different resource files to generate a binary format optimized for the Android platform (flat). In addition to the content of the original resource file, the FLAT file also contains information such as the source and type of the resource. Such a file contains all the information required by the resource and is coupled to other dependencies.

At the beginning of this article, we have the following questions:

Java files need to be compiled to generate .class files. I can understand this, but what does the resource file compilation do? Why do we need to compile resources?

So, the answer to this article is: AAPT2 compiles the resource file into a FLAT file when compiling, and from the file structure of the resource item, it can be known that part of the data in the FLAT file is the original resource content, and part is the relevant information of the file. Through compilation, the generated intermediate file contains more comprehensive information and can be used for incremental compilation. In addition, some information on the Internet also indicates that binary resources are smaller and load faster.

AAPT2 compiles resource files into FLAT files through compilation, and then generates R files and resource tables through linking. Due to space issues, the linking process will be analyzed in the next article.

Four, reference documents

1.https://juejin.cn

2.https://github.com

3.https://booster.johnsonlee.io

Author: vivo Internet front-end team-Shi Xiang