微信资源混淆AndResGuard源码解析(三)
前言
前两篇文章主要记录了，resources.arsc的读和写的流程和细节，这一篇文章，记录的主要就是在resources.arsc写回操作完成之后，构建apk的流程和细节
[Main-> buildApk]
private void buildApk(
ApkDecoder decoder, File apkFile, File outputFile, InputParam.SignatureType signatureType, int minSDKVersion)
throws Exception {
ResourceApkBuilder builder = new ResourceApkBuilder(config);
String apkBasename = apkFile.getName();
apkBasename = apkBasename.substring(0, apkBasename.indexOf(".apk"));
builder.setOutDir(mOutDir, apkBasename, outputFile);
switch (signatureType) {
case SchemaV1:
builder.buildApkWithV1sign(decoder.getCompressData());
break;
case SchemaV2:
builder.buildApkWithV2sign(decoder.getCompressData(), minSDKVersion);
break;
}
}
- 创建一个ResourceApkBuilder对象
- 然后分别根据V1和V2签名，用ResourceApkBuilder构建出apk，特别关注的就是decoder.getCompressData()，这里包含了一些压缩处理是之前还没有太多留意的
[ResourceApkBuilder-> buildApkWithV1sign]
public void buildApkWithV1sign(HashMap<String, Integer> compressData) throws IOException, InterruptedException {
insureFileNameV1();
generalUnsignApk(compressData);
signApkV1(mUnSignedApk, mSignedApk);
use7zApk(compressData, mSignedApk, mSignedWith7ZipApk);
alignApks();
copyFinalApkV1();
}
[ResourceApkBuilder-> insureFileNameV1]
private void insureFileNameV1() {
mUnSignedApk = new File(mOutDir.getAbsolutePath(), mApkName + "_unsigned.apk");
mSignedWith7ZipApk = new File(mOutDir.getAbsolutePath(), mApkName + "_signed_7zip.apk");
mSignedApk = new File(mOutDir.getAbsolutePath(), mApkName + "_signed.apk");
mAlignedApk = new File(mOutDir.getAbsolutePath(), mApkName + "_signed_aligned.apk");
mAlignedWith7ZipApk = new File(mOutDir.getAbsolutePath(), mApkName + "_signed_7zip_aligned.apk");
m7zipOutPutDir = new File(mOutDir.getAbsolutePath(), TypedValue.OUT_7ZIP_FILE_PATH);
}
- 这个方法其实就是构建出构建所需的所有文件
[ResourceApkBuilder-> generalUnsignApk]
File tempOutDir = new File(mOutDir.getAbsolutePath(), TypedValue.UNZIP_FILE_PATH);
File[] unzipFiles = tempOutDir.listFiles();
assert unzipFiles != null;
List<File> collectFiles = new ArrayList<>();
for (File f : unzipFiles) {
String name = f.getName();
if (name.equals("res") || name.equals("resources.arsc")) {
continue;
} else if (name.equals(config.mMetaName)) {
addNonSignatureFiles(collectFiles, f);
continue;
}
collectFiles.add(f);
}
- 这里可以看到首先会构建一个临时文件，然后把遍历解压文件，如果是res目录或者是resources.arsc就不予处理
- 如果遇到的是签名文件夹，就调用addNonSignatureFiles进行操作
- 最终都会把除了res目录和resources.arsc以及的文件保存在collectFiles中，
[ResourceApkBuilder-> addNonSignatureFiles]
private void addNonSignatureFiles(List<File> collectFiles, File metaFolder) {
File[] metaFiles = metaFolder.listFiles();
if (metaFiles != null) {
for (File metaFile : metaFiles) {
String metaFileName = metaFile.getName();
// Ignore signature files
if (!metaFileName.endsWith(".MF") && !metaFileName.endsWith(".RSA") && !metaFileName.endsWith(".SF")) {
System.out.println(String.format("add meta file %s", metaFile.getAbsolutePath()));
collectFiles.add(metaFile);
}
}
}
}
- 在这个方法里面，其实collectFiles添加进去的也是和签名无关的文件
- 所以总结起来collectFiles这个容器中存放的就是除签名，arsc，res之外的所有文件
- 剩下就是看一下什么时候用到这个collectFiles了
[ResourceApkBuilder-> generalUnsignApk]
File destResDir = new File(mOutDir.getAbsolutePath(), "res");
//添加修改后的res文件
if (!config.mKeepRoot && FileOperation.getlist(destResDir) == 0) {
destResDir = new File(mOutDir.getAbsolutePath(), TypedValue.RES_FILE_PATH);
}
File rawResDir = new File(tempOutDir.getAbsolutePath() + File.separator + "res");
collectFiles.add(destResDir);
File rawARSCFile = new File(mOutDir.getAbsolutePath() + File.separator + "resources.arsc");
collectFiles.add(rawARSCFile);
FileOperation.zipFiles(collectFiles, tempOutDir, mUnSignedApk, compressData);
- 这里其实说实话一开始看是有点懵的，这主要是因为文件的关系没有理清，其实这个作者我觉得做得是挺高明的一点就是避免了参数传递，因为全过程中涉及文件的操作比较多，如果把确认好的文件到处传这样方法参数比较臃肿，我猜想这个作者当时的想法应该就是全局制定一些条约，根据文件名，去在不同的类中拿到想要的文件
- destResDir：这个就是混淆之后的r文件夹，这里这个判断的就是判断res文件夹里是否有文件，有的话就说明原来储存的资源的文件夹就是res，也就是是根目录不混淆的情况，如果没有的话，就说明根目录就是r，就是经过混淆的情况
- 然后就把resources.arsc和destResDir添加到之前的collectFiles中
- 然后就意味着apk的文件已经收集完成，就进行压缩操作，在这个压缩操作中需要4个参数，收集完成的apk文件，解压目录，一开始设定好的UnSignedApk文件，极度关键的compressData，关注它是如何实现压缩的
[FileOperation-> zipFiles]
public static void zipFiles(
Collection<File> resFileList, File baseFolder, File zipFile, HashMap<String, Integer> compressData)
throws IOException {
ZipOutputStream zipOut = new ZipOutputStream(new BufferedOutputStream(new FileOutputStream(zipFile), BUFFER));
for (File resFile : resFileList) {
if (resFile.exists()) {
if (resFile.getAbsolutePath().contains(baseFolder.getAbsolutePath())) {
String relativePath = baseFolder.toURI().relativize(resFile.getParentFile().toURI()).getPath();
// remove slash at end of relativePath
if (relativePath.length() > 1) {
relativePath = relativePath.substring(0, relativePath.length() - 1);
} else {
relativePath = "";
}
zipFile(resFile, zipOut, relativePath, compressData);
} else {
zipFile(resFile, zipOut, "", compressData);
}
}
}
zipOut.close();
}
- 这个方法暂时没用到compressData
- 但是这里拿到了一个relativePath，这个relativePath是什么？就是当前文件和源目录底下相隔的目录字符串
[FileOperation-> zipFile]
private static void zipFile(
File resFile, ZipOutputStream zipout, String rootpath, HashMap<String, Integer> compressData) throws IOException {
rootpath = rootpath + (rootpath.trim().length() == 0 ? "" : File.separator) + resFile.getName();
if (resFile.isDirectory()) {
File[] fileList = resFile.listFiles();
for (File file : fileList) {
zipFile(file, zipout, rootpath, compressData);
}
} else {
final byte[] fileContents = readContents(resFile);
//这里需要强转成linux格式，果然坑！！
if (rootpath.contains("\\")) {
rootpath = rootpath.replace("\\", "/");
}
if (!compressData.containsKey(rootpath)) {
System.err.printf(String.format("do not have the compress data path =%s in resource.asrc\n", rootpath));
//throw new IOException(String.format("do not have the compress data path=%s", rootpath));
return;
}
int compressMethod = compressData.get(rootpath);
ZipEntry entry = new ZipEntry(rootpath);
if (compressMethod == ZipEntry.DEFLATED) {
entry.setMethod(ZipEntry.DEFLATED);
} else {
entry.setMethod(ZipEntry.STORED);
entry.setSize(fileContents.length);
final CRC32 checksumCalculator = new CRC32();
checksumCalculator.update(fileContents);
entry.setCrc(checksumCalculator.getValue());
}
zipout.putNextEntry(entry);
zipout.write(fileContents);
zipout.flush();
zipout.closeEntry();
}
}
- 首先把资源文件的内存读出来保存在byte数组fileContent中
- 可以看到，这里的逻辑就是判断compressData是否包含rootpath，如果没有就直接返回了
- 读到这里有个问题，这个compressData是什么时候put东西进去的？目测是之前分析resources.arsc读取流程的时候漏掉了
[ARSCDecoder-> readValue]
//这里用的是linux的分隔符
HashMap<String, Integer> compressData = mApkDecoder.getCompressData();
if (compressData.containsKey(raw)) {
compressData.put(result, compressData.get(raw));
} else {
System.err.printf("can not find the compress dataresFile=%s\n", raw);
}
- 继续跟踪回mApkDecoder中的compressData
[ApkDecoder-> ensureFilePath]
private void ensureFilePath() throws IOException {
mCompressData = FileOperation.unZipAPk(apkFile.getAbsoluteFile().getAbsolutePath(), unZipDest);
dealWithCompressConfig();
}
[FileOperation-> unZipAPk]
public static HashMap<String, Integer> unZipAPk(String fileName, String filePath) throws IOException {
checkDirectory(filePath);
ZipFile zipFile = new ZipFile(fileName);
Enumeration emu = zipFile.entries();
HashMap<String, Integer> compress = new HashMap<>();
try {
while (emu.hasMoreElements()) {
ZipEntry entry = (ZipEntry) emu.nextElement();
if (entry.isDirectory()) {
new File(filePath, entry.getName()).mkdirs();
continue;
}
BufferedInputStream bis = new BufferedInputStream(zipFile.getInputStream(entry));
File file = new File(filePath + File.separator + entry.getName());
File parent = file.getParentFile();
if (parent != null && (!parent.exists())) {
parent.mkdirs();
}
//要用linux的斜杠
String compatibaleresult = entry.getName();
if (compatibaleresult.contains("\\")) {
compatibaleresult = compatibaleresult.replace("\\", "/");
}
compress.put(compatibaleresult, entry.getMethod());
FileOutputStream fos = new FileOutputStream(file);
BufferedOutputStream bos = new BufferedOutputStream(fos, BUFFER);
byte[] buf = new byte[BUFFER];
int len;
while ((len = bis.read(buf, 0, BUFFER)) != -1) {
fos.write(buf, 0, len);
}
bos.flush();
bos.close();
bis.close();
}
} finally {
zipFile.close();
}
return compress;
}
- 首先第一步对输出目录进行检查，如果存在就删除重建
- 然后遍历apk中的Entry
- 在compress里面存的是修改后的entryName对应entry压缩方法的映射
[ApkDecoder-> dealWithCompressConfig]
private void dealWithCompressConfig() {
if (config.mUseCompress) {
HashSet<Pattern> patterns = config.mCompressPatterns;
if (!patterns.isEmpty()) {
for (Entry<String, Integer> entry : mCompressData.entrySet()) {
String name = entry.getKey();
for (Iterator<Pattern> it = patterns.iterator(); it.hasNext(); ) {
Pattern p = it.next();
if (p.matcher(name).matches()) {
mCompressData.put(name, TypedValue.ZIP_DEFLATED);
}
}
}
}
}
}
- 看到这里真的有一种神清气爽的感觉
- 这里第一步就是从config中拿到mCompressPatterns，而config中的mCompressPatterns的来源就是来自gradle extension
- 然后接下来只需要遍历一次mCompressData，然后把获得的entry名称和正则表达式对比，如果是match的，那就put进ZIP_DEFLATED这个value
- 所以实际上压缩就是吧entry的compressMethod设置成TypedValue.ZIP_DEFLATED，也就是说其实AndResGuard压缩的本质就是把压缩方式设置成DEFLATED，让它可以压缩存储，而不是打包归档
[ResourceApkBuilder-> generalUnsignApk]
[ResourceApkBuilder-> buildApkWithV1sign]
[ResourceApkBuilder-> signApkV1]
private void signApkV1(File unSignedApk, File signedApk) throws IOException, InterruptedException {
if (config.mUseSignAPK) {
System.out.printf("signing apk: %s\n", signedApk.getName());
if (signedApk.exists()) {
signedApk.delete();
}
signWithV1sign(unSignedApk, signedApk);
if (!signedApk.exists()) {
throw new IOException("Can't Generate signed APK. Plz check your v1sign info is correct.");
}
}
}
[ResourceApkBuilder-> signWithV1sign]
private void signWithV1sign(File unSignedApk, File signedApk) throws IOException, InterruptedException {
String signatureAlgorithm = "MD5withRSA";
try {
signatureAlgorithm = getSignatureAlgorithm(config.digestAlg);
} catch (Exception e) {
e.printStackTrace();
}
String[] argv = {
"jarsigner",
"-sigalg",
signatureAlgorithm,
"-digestalg",
config.digestAlg,
"-keystore",
config.mSignatureFile.getAbsolutePath(),
"-storepass",
config.mStorePass,
"-keypass",
config.mKeyPass,
"-signedjar",
signedApk.getAbsolutePath(),
unSignedApk.getAbsolutePath(),
config.mStoreAlias
};
Utils.runExec(argv);
}
- 可以看到签名的方式就是采用命令行工具执行jarsigner命令
[ResourceApkBuilder-> buildApkWithV1sign]
[ResourceApkBuilder-> use7zApk]
private boolean use7zApk(HashMap<String, Integer> compressData, File originalAPK, File outputAPK)
throws IOException, InterruptedException {
FileOperation.unZipAPk(originalAPK.getAbsolutePath(), m7zipOutPutDir.getAbsolutePath());
//首先一次性生成一个全部都是压缩的安装包
generalRaw7zip(outputAPK);
ArrayList<String> storedFiles = new ArrayList<>();
for (String name : compressData.keySet()) {
File file = new File(m7zipOutPutDir.getAbsolutePath(), name);
if (!file.exists()) {
continue;
}
int method = compressData.get(name);
if (method == TypedValue.ZIP_STORED) {
storedFiles.add(name);
}
}
addStoredFileIn7Zip(storedFiles, outputAPK);
return true;
}
- 首先第一步就是把原本的签名apk解压
- 调用generalRaw7zip方法
- 对于不压缩的文件文件用一个容器储存起来
- 调用addStoredFileIn7Zip方法
[ResourceApkBuilder-> generalRaw7zip]
private void generalRaw7zip(File outSevenZipApk) throws IOException, InterruptedException {
String outPath = m7zipOutPutDir.getAbsoluteFile().getAbsolutePath();
String path = outPath + File.separator + "*";
String cmd = Utils.isPresent(config.m7zipPath) ? config.m7zipPath : TypedValue.COMMAND_7ZIP;
Utils.runCmd(cmd, "a", "-tzip", outSevenZipApk.getAbsolutePath(), path, "-mx9");
}
- 采用命令行工具实现7zip压缩
[ResourceApkBuilder-> addStoredFileIn7Zip]
private void addStoredFileIn7Zip(ArrayList<String> storedFiles, File outSevenZipAPK)
throws IOException, InterruptedException {
String storedParentName = mOutDir.getAbsolutePath() + File.separator + "storefiles" + File.separator;
String outputName = m7zipOutPutDir.getAbsolutePath() + File.separator;
for (String name : storedFiles) {
FileOperation.copyFileUsingStream(new File(outputName + name), new File(storedParentName + name));
}
storedParentName = storedParentName + File.separator + "*";
String cmd = Utils.isPresent(config.m7zipPath) ? config.m7zipPath : TypedValue.COMMAND_7ZIP;
Utils.runCmd(cmd, "a", "-tzip", outSevenZipAPK.getAbsolutePath(), storedParentName, "-mx0");
}
- 这里的任务就是把刚刚设计stored压缩方式的文件添加到7zip压缩包中
[ResourceApkBuilder-> buildApkWithV1sign]
[ResourceApkBuilder-> alignApks]
private void alignApks() throws IOException, InterruptedException {
if (mSignedApk.exists()) {
alignApk(mSignedApk, mAlignedApk);
}
if (mSignedWith7ZipApk.exists()) {
alignApk(mSignedWith7ZipApk, mAlignedWith7ZipApk);
}
}
[ResourceApkBuilder-> alignApk]
private void alignApk(File before, File after) throws IOException, InterruptedException {
String cmd = Utils.isPresent(config.mZipalignPath) ? config.mZipalignPath : TypedValue.COMMAND_ZIPALIGIN;
Utils.runCmd(cmd, "4", before.getAbsolutePath(), after.getAbsolutePath());
}
- 也是使用命令行工具进行apk对齐
阶段小结
整个v1签名的apk生成的过程走完了，总结起来这里包括align和7zip还有v1签名都是采用命令行工具完成的，而压缩的话则是通过设置ZipEntry的method实现的
[Main-> buildApk]
[ResourceApkBuilder-> buildApkWithV2sign]
public void buildApkWithV2sign(HashMap<String, Integer> compressData, int minSDKVersion) throws Exception {
insureFileNameV2();
generalUnsignApk(compressData);
if (use7zApk(compressData, mUnSignedApk, m7ZipApk)) {
alignApk(m7ZipApk, mAlignedApk);
} else {
alignApk(mUnSignedApk, mAlignedApk);
}
/*
* Caution: If you sign your app using APK Signature Scheme v2 and make further changes to the app,
* the app's signature is invalidated.
* For this reason, use tools such as zipalign before signing your app using APK Signature Scheme v2, not after.
**/
signApkV2(mAlignedApk, mSignedApk, minSDKVersion);
copyFinalApkV2();
}
- v2签名的处理流程和v1签名的处理流程大同小异
- 不一样的是v2签名的逻辑中align的操作是放在v2签名之前的，这是因为v2签名保护的是zip文件的1、3、4部分后如果进行align操作的话就会被视为改变的apk结构，然后就会签名校验失败
[ResourceApkBuilder-> signApkV2]
[ResourceApkBuilder-> signWithV2sign]
private void signWithV2sign(File unSignedApk, File signedApk, int minSDKVersion) throws Exception {
String[] params = new String[] {
"sign",
"--ks",
config.mSignatureFile.getAbsolutePath(),
"--ks-pass",
"pass:" + config.mStorePass,
"--min-sdk-version",
String.valueOf(minSDKVersion),
"--ks-key-alias",
config.mStoreAlias,
"--key-pass",
"pass:" + config.mKeyPass,
"--out",
signedApk.getAbsolutePath(),
unSignedApk.getAbsolutePath()
};
ApkSignerTool.main(params);
}
- 采用android开源的ApkSignerTool实现v2签名
阶段小结
来到这里，其实整个构建apk的流程都走完，但是这篇文章还没有结束，因为其实还有一个部分是没有看的AndResGuard是如何实现资源合并的
[ARSCDecoder-> readValue]
MergeDuplicatedResInfo filterInfo = null;
boolean mergeDuplicatedRes = mApkDecoder.getConfig().mMergeDuplicatedRes;
if (mergeDuplicatedRes) {
filterInfo = mergeDuplicated(resRawFile, resDestFile, compatibaleraw, result);
if (filterInfo != null) {
resDestFile = new File(filterInfo.filePath);
result = filterInfo.fileName;
}
}
[ApkDecoder-> mergeDuplicated]
private MergeDuplicatedResInfo mergeDuplicated(File resRawFile, File resDestFile, String compatibaleraw, String result) throws IOException {
MergeDuplicatedResInfo filterInfo = null;
// 1
List<MergeDuplicatedResInfo> mergeDuplicatedResInfoList = mMergeDuplicatedResInfoData.get(resRawFile.length());
if (mergeDuplicatedResInfoList != null) {
// ......
}else {
MergeDuplicatedResInfo info = new MergeDuplicatedResInfo.Builder()
.setFileName(result)
.setFilePath(resDestFile.getAbsolutePath())
.setOriginalName(compatibaleraw)
.create();
info.fileName = result;
info.filePath = resDestFile.getAbsolutePath();
info.originalName = compatibaleraw;
if (mergeDuplicatedResInfoList == null) {
mergeDuplicatedResInfoList = new ArrayList<>();
mMergeDuplicatedResInfoData.put(resRawFile.length(), mergeDuplicatedResInfoList);
}
mergeDuplicatedResInfoList.add(info);
}
return filterInfo;
- 注释1的地方有一个成员mMergeDuplicatedResInfoData，但是所有的get和put的操作都是在这个方式执行的，所以这里肯定是不命中的，所以先看不命中的逻辑
- 首先是构建一个MergeDuplicatedResInfo对象，但是这里有点迷，不知道为什么builder模式要设置一次值然后后面又设置一次，然后就是把这个对象存进一个list中
- 然后就根据原来的资源文件的大小作为key，存放进mMergeDuplicatedResInfoData这个成员里面
- 所以很明显，文件长度是一个判断重复的其中一个标准
if (mergeDuplicatedResInfoList != null) {
for (MergeDuplicatedResInfo mergeDuplicatedResInfo : mergeDuplicatedResInfoList) {
if (mergeDuplicatedResInfo.md5 == null) {
mergeDuplicatedResInfo.md5 = Md5Util.getMD5Str(new File(mergeDuplicatedResInfo.filePath));
}
String resRawFileMd5 = Md5Util.getMD5Str(resRawFile);
if (!resRawFileMd5.isEmpty() && resRawFileMd5.equals(mergeDuplicatedResInfo.md5)) {
filterInfo = mergeDuplicatedResInfo;
filterInfo.md5 = resRawFileMd5;
break;
}
}
}
if (filterInfo != null) {
generalFilterResIDMapping(compatibaleraw, result, filterInfo.originalName, filterInfo.fileName, resRawFile.length());
mMergeDuplicatedResCount++;
mMergeDuplicatedResTotalSize += resRawFile.length();
- 这个就是缓存命中的的情况，它的逻辑其实也就是计算当前文件的md5的值以及和当前文件长度一致的文件列表的md5的值，如果是一致的就说明是重复了
- 然后剩下的就是一些mapping写入和size记录的工作了
- 所以资源查重的标准是两个点，一个是文件长度是否一致，另一个就是md5的值是否相同
