Android CTS 测试研究之二

作者：宋立新

Email ： zjujoe@yahoo.com

前言

继续较深入了解 CTS 的细节

先研究一下具体用法

命令行下敲 cts 进入命令行 ,

先看一下帮助

Help 一下

cts_host > help

Usage: command options

Avaiable commands and options:

Host:

help: show this message

exit: exit cts command line

Plan:

ls --plan: list available plans

ls --plan plan_name: list contents of the plan with specified name

add --plan plan_name: add a new plan with specified name

add --derivedplan plan_name -s/--session session_id -r/--result result_type: derive a plan from the given session

rm --plan plan_name/all: remove a plan or all plans from repository

start --plan test_plan_name: run a test plan

start --plan test_plan_name -d/--device device_ID: run a test plan using the specified device

start --plan test_plan_name -t/--test test_name: run a specific test

start --plan test_plan_name -p/--package java_package_name: run a specific java package

start --plan test_plan_name -t/--test test_name -d/--device device_ID: run a specific test using the specified device

start --plan test_plan_name -p/--package java_package_name -d/--device device_ID: run a specific java package using the specified device

Package:

ls -p/--package: list available packages

ls -p/--package package_name: list contents of the package with specified name

add -p/--package root: add packages from root to repository

rm -p/--package package_name/all: remove a package or all packages from repository

Result:

ls -r/--result: list all result of sessions

ls -r/--result -s/--session session_id: list detail case result of a specified session

ls -r/--result [pass/fail/notExecuted/timeout] -s/--session session_id: list detail cases of a specified session by the specified result.

History:

history/h: list all commands in command history

history/h count: list the latest count records in command history

history/h -e num: run the command designated by 'num' in command history

Device:

ls -d/--device: list available devices

列出所有的 plan

cts_host > ls --plan

List of plans (8 in total):

CTS

Signature

AppSecurity

Performance

RefApp

Java

Android

查看某 plan 的内容

cts_host > ls --plan Android

Packages of plan Android (29 in total):

=================================

android.apidemos.cts

android.accessibilityservice

android.accounts

android.app

android.bluetooth

android.content

android.database

android.dpi

android.dpi2

android.example

android.gesture

android.graphics

android.hardware

android.jni

android.location

android.media

android.net

android.os

android.permission2

android.permission

android.provider

android.speech

android.telephony

android.text

android.util

android.view

android.webkit

android.widget

android.tests.appsecurity

添加一个新的 plan

cts_host > add --plan marvell

[Choose package] SignatureTest: select[Y], reject[n], or choose suite in it[m]? [Y/n/m] Y

Invalid input. Please chose 'y', 'n', or 'm' (default is 'y')

[Choose package] SignatureTest: select[Y], reject[n], or choose suite in it[m]? [Y/n/m] y

[Choose package] android.accessibilityservice: select[Y], reject[n], or choose suite in it[m]? [Y/n/m] y

[Choose package] android.accounts: select[Y], reject[n], or choose suite in it[m]? [Y/n/m] y

[Choose package] android.apidemos.cts: select[Y], reject[n], or choose suite in it[m]? [Y/n/m]

[Choose package] android.app: select[Y], reject[n], or choose suite in it[m]? [Y/n/m] y

。。。。

[Choose package] android.widget: select[Y], reject[n], or choose suite in it[m]? [Y/n/m]

cts_host > ls --plan marvell

The following package(s) contained in plan marvell have been removed:

SignatureTest

Packages of plan marvell (55 in total):

=================================

android.accessibilityservice

android.accounts

android.apidemos.cts

android.app

。。。

android.widget

删除一个 plan

cts_host > rm --plan marvell

执行一个 plan

cts_host > start --plan Performance

start test plan Performance

==============================================================

Test package: android.performance2

android.performance2.cts.AppStartup#testStartup........................................................................................................................................................(timeout)

==============================================================

CTS_INFO >>> Max ADB operations reached. Restarting ADB...

CTS_INFO >>> Restarting device ...

CTS_INFO >>> Restart complete.

==============================================================

Test package: android.performance3

android.performance3.cts.AppStartup#testStartup..............(pass)

CTS_INFO >>> Max ADB operations reached. Restarting ADB...

CTS_INFO >>> Restarting device ...

CTS_INFO >>> Restart complete.

==============================================================

Test package: android.performance4

android.performance4.cts.AppStartup#testStartup.......(pass)

CTS_INFO >>> Max ADB operations reached. Restarting ADB...

CTS_INFO >>> Restarting device ...

CTS_INFO >>> Restart complete.

==============================================================

Test package: android.performance5

android.performance5.cts.AppStartup#testStartup........(fail)

junit.framework.AssertionFailedError: App Took too long to startup: 715 650 at android.performance5.cts.AppStartup.testStartup(AppStartup.java:67)

at android.performance5.cts.AppStartup.testStartup(AppStartup.java:67)

at java.lang.reflect.Method.invokeNative(Native Method)

at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:205)

at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:195)

at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)

at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)

at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:430)

at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)

at android.performance5.cts.AppStartup.testStartup(AppStartup.java:67)

at java.lang.reflect.Method.invokeNative(Native Method)

at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:205)

at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:195)

at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)

at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)

at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:430)

at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)

at android.app.Instrumentation.checkStartActivityResult(Instrumentation.java:1404)

at android.app.Instrumentation.execStartActivity(Instrumentation.java:1378)

at android.app.ApplicationContext.startActivity(ApplicationContext.java:555)

at android.performance.cts.MultiAppStartupTest.launchActivity(MultiAppStartupTest.java:52)

at android.performance.cts.MultiAppStartupTest.testMultipleApps(MultiAppStartupTest.java:89)

at java.lang.reflect.Method.invokeNative(Native Method)

at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:205)

at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:195)

at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:169)

at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:154)

at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:430)

at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1447)

Test summary: pass=2 fail=2 timeOut=1 notExecuted=0 Total=5

Time: 635.927s

查看有多少个包

cts_host > ls -p

Available packages (56 in total):

android.util

android.hardware

。。。

android.core.tests.luni.net

android.jni

android.core.vm-tests

查看某个包

cts_host > ls -p android.net

Test suites (3 in total):

android.net.cts

android.net.wifi.cts

android.net.http.cts

查看测试结果

cts_host > ls -r

List of all results:

Session Test result Start time End time Test plan name

Pass Fail Timeout NotExecuted

1 2 2 1 0 2010.06.01 16:18:03 2010.06.01 16:28:39 Performance

2 1 0 0 0 2010.06.01 16:39:38 2010.06.01 16:41:33 Signature

研究 cts 生成文件

cts 脚本文件位于： ~/mydroid/out/host/linux-x86/bin ，它是一个 100 行不到的脚本，最终执行：

java -Xmx512M

-cp ./../framework/ddmlib.jar:./../framework/junit.jar:./../framework/hosttestlib.jar:./../framework/cts.jar:./cts.jar

-DHOST_CONFIG=./../cts/android-cts/repository/host_config.xml

com.android.cts.TestHost

可见， CTS 的 Java 主程序为： com.android.cts.TestHost ，同时使用了配置文件： host_config.xml.

两个 cts.jar: ./out/host/linux-x86/cts/android-cts/tools/cts.jar ./out/host/linux-x86/framework/cts.jar 相同。

cts 使用的目录则为： ~/mydroid/out/host/linux-x86/cts

文件列表 1

songlixin @zjujoe-desktop:~/mydroid/out/host/linux-x86/cts$ tree -L 3

├── all_cts_core_files_stamp

├── all_cts_files_stamp

├── android-cts

│ ├── docs

│ ├── repository

│ │ ├── host_config.xml

│ │ ├── log_2010.06.01_16.02.21_.txt

│ │ ├── log_2010.06.01_16.12.22_.txt

│ │ ├── log_2010.06.01_16.38.30_.txt

│ │ ├── log_2010.06.01_16.59.50_.txt

│ │ ├── plans

│ │ ├── results

│ │ └── testcases

│ └── tools

│ ├── cts.jar

│ ├── hosttestlib.jar

│ ├── junit.jar

│ └── startcts

├── android-cts.zip

└── temp

└── description.xml

我们关心的主要是 repository 目录。

文件列表 2

songlixin @zjujoe-desktop:~/mydroid/out/host/linux-x86/cts$ tree android-cts/repository/

android-cts/repository/

├── host_config.xml

├── plans

│ ├── Android.xml

│ ├── AppSecurity.xml

│ ├── CTS.xml

│ ├── Java.xml

│ ├── Performance.xml

│ ├── RefApp.xml

│ ├── Signature.xml

│ └── VM.xml

├── results

│ ├── 2010.06.01_16.18.03

│ │ ├── cts_result.css

│ │ ├── cts_result.xsl

│ │ ├── logo.gif

│ │ ├── newrule-green.png

│ │ └── testResult.xml

│ ├── 2010.06.01_16.18.03.zip

│ ├── 2010.06.01_16.39.38

│ │ ├── cts_result.css

│ │ ├── cts_result.xsl

│ │ ├── logo.gif

│ │ ├── newrule-green.png

│ │ └── testResult.xml

│ ├── 2010.06.01_16.39.38.zip

│ ├── cts_result.css

│ ├── cts_result.xsl

│ ├── logo.gif

│ └── newrule-green.png

└── testcases

├── android.core.tests.annotation.apk

├── android.core.tests.annotation.xml

├── android.core.tests.archive.apk

。。。。。。。。。。。。。。。。。。

├── CtsWidgetTestCases.xml

├── SignatureTest.apk

├── SignatureTest.xml

└── TestDeviceSetup.apk

5 directories, 156 files

可见 plans 目录下为以 xml 格式定义的测试方案

results 目录为测试结果。

testcases 目录则为一个个测试用例包

host_config.xml 为配置文件

研究 cts 源文件

从根目录开始

Cts 的 source 包位于 android 根目录下：

songlixin @zjujoe-desktop:~/mydroid/cts$ tree -L 2

├── Android.mk

├── tests

│ ├── accessibilityservice

│ ├── AndroidManifest.xml

│ ├── Android.mk

│ ├── ApiDemosReferenceTest

│ ├── appsecurity-tests

│ ├── assets

│ ├── config_demo

│ ├── core

│ ├── ProcessTest

│ ├── res

│ ├── SignatureTest

│ ├── src

│ ├── tests

│ └── vm-tests

└── tools

├── Android.mk

├── annotation-helper

├── cts-reference-app-lib

├── dasm

├── device-setup

├── dex-tools

├── dx-tests

├── host

├── signature-tools

├── spec-progress

├── test-progress

├── test-progress-new

├── utils

└── vm-tests

根目录下的 make file 非常简单：

songlixin @zjujoe-desktop:~/mydroid/cts$ cat Android.mk

include $(call all-subdir-makefiles)

看来主要内容在两个子目录下。

子目录 1 ： tools

根目录下的 Android.mk 与其父目录相同。

各子目录代表不同的工具，比如 dasm 编译后的位置是： ./out/host/linux-x86/bin/dasm

这里就不仔细分析这些工具了。

子目录 2 ： tests

看一下根目录下的 Android.mk:

LOCAL_PATH:= $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE_TAGS := tests

LOCAL_SRC_FILES := $(call all-java-files-under, src)/

$(call all-java-files-under, core/runner/src)/

src/android/app/cts/ISecondary.aidl/

src/android/os/cts/IEmptyService.aidl

LOCAL_JAVA_LIBRARIES := android.test.runner

# Resource unit tests use a private locale and some densities

LOCAL_AAPT_FLAGS = -c xx_YY -c cs -c 32dpi -c 240dpi -c 160dpi

LOCAL_PACKAGE_NAME := CtsTestStubs

include $(BUILD_PACKAGE)

# Build the test APK using its own makefile, and any other CTS-related packages

include $(call all-makefiles-under,$(LOCAL_PATH))

所有 src 及 core/runner/src 下的 java 文件被编译成：

./out/host/linux-x86/cts/android-cts/repository/testcases/CtsTestStubs.apk

剩下的就是很多子目录以及子目录的子目录会被编译成 *.apk

Tests 目录下每个子目录包括一个 test suite.

songlixin @zjujoe-desktop:~/mydroid/cts/tests/tests$ tree -L 1

├── accessibilityservice

├── accounts

├── Android.mk

。。。

├── webkit

└── widget

Core 目录下每个子目录包含一个 test suite, 但是他们的代码在其他地方

songlixin @zjujoe-desktop:~/mydroid/cts/tests/core$ tree -L 2

├── Android.mk

├── annotation

│ ├── AndroidManifest.xml

│ └── Android.mk

├── archive

│ ├── AndroidManifest.xml

│ └── Android.mk

。。。

├── text

│ ├── AndroidManifest.xml

│ └── Android.mk

├── xml

│ ├── AndroidManifest.xml

│ └── Android.mk

└── xnet

├── AndroidManifest.xml

└── Android.mk

appsecurity-tests/test-apps 目录下每个子目录包括一个 test case. 都特别简单 .

songlixin @zjujoe-desktop:~/mydroid/cts/tests/appsecurity-tests/test-apps$ tree -L 1

├── Android.mk

├── AppAccessData

├── AppWithData

├── InstrumentationAppDiffCert

├── PermissionDeclareApp

├── SharedUidInstall

├── SharedUidInstallDiffCert

├── SimpleAppInstall

├── SimpleAppInstallDiffCert

├── TargetInstrumentationApp

└── UsePermissionDiffCert

总结

Cts 其实就是基于 instrumentation test, Test case 有很多类型，不同的实现方式。对我们来说，需要：

1 ）能够执行现有的 test case

2 ）能够模仿 api demo test case 写一些比较简单的：

可以参考： http://www.netmite.com/ /

android/mydroid/cupcake/development/pdk/docs/instrumentation_testing.html

Android CTS 测试研究之二

Android CTS 测试研究之二

前言

先研究一下具体用法

先看一下帮助

列出所有的 plan

查看某 plan 的内容

添加一个新的 plan

删除一个 plan

执行一个 plan

查看有多少个包

查看某个包

查看测试结果

研究 cts 生成文件

文件列表 1

文件列表 2

研究 cts 源文件

从根目录开始

子目录 1 ： tools

子目录 2 ： tests

总结

更多相关文章

随机推荐