【知识】【python】【adb】uiautomator2 包使用指南

python 通过 uiautomator2 包操作安卓设备界面的各种方法

安装及 init

1
pip install uiautomator2
1
2
3
4
5
6
7
8
9
10
import uiautomator2 as ui2

# 1. wifi 连
d = ui2.connect('10.0.0.1')
# 2. usb 连
d = ui2.connect('123456f')
# 3. adb wifi 连
d = u2.connect_adb_wifi("10.0.0.1:5555")

print(d.info())

API 合集

APP 操作

  1. 安装 app
    d.app_install('http://some-domain.com/some.apk')

  2. 启动 app

    1
    2
    3
    4
    5
    6
    7
    8
    9
    # 默认的这种方法是先通过atx-agent解析apk包的mainActivity，然后调用am start -n $package/$activity启动
    d.app_start("com.example.hello_world")

    # 使用 monkey -p com.example.hello_world -c android.intent.category.LAUNCHER 1 启动
    # 这种方法有个副作用，它自动会将手机的旋转锁定给关掉
    d.app_start("com.example.hello_world", use_monkey=True) # start with package name

    # 通过指定main activity的方式启动应用，等价于调用am start -n com.example.hello_world/.MainActivity
    d.app_start("com.example.hello_world", ".MainActivity")

  3. 停止 app 运行

  • 单个 app

    1
    2
    3
    4
    # equivalent to `am force-stop`, thus you could lose data
    d.app_stop("com.example.hello_world") 
    # equivalent to `pm clear`
    d.app_clear('com.example.hello_world')

  • 所有 app

    1
    2
    3
    4
    # stop all
    d.app_stop_all()
    # stop all app except for com.examples.demo
    d.app_stop_all(excludes=['com.examples.demo'])
  1. 获取 app 信息

  • 指定 app

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    d.app_info("com.examples.demo")
    # expect output
    #{
    #    "mainActivity": "com.github.uiautomator.MainActivity",
    #    "label": "ATX",
    #    "versionName": "1.1.7",
    #    "versionCode": 1001007,
    #    "size":1760809
    #}

    # save app icon
    img = d.app_icon("com.examples.demo")
    img.save("icon.png")

  • 正在运行的 app

    1
    2
    3
    d.app_list_running()
    # expect output
    # ["com.xxxx.xxxx", "com.github.uiautomator", "xxxx"]
1
2
3
4
5
6
7
8
pid = d.app_wait("com.example.android") # 等待应用运行, return pid(int)
if not pid:
    print("com.example.android is not running")
else:
    print("com.example.android pid is %d" % pid)

d.app_wait("com.example.android", front=True) # 等待应用前台运行
d.app_wait("com.example.android", timeout=20.0) # 最长等待时间20s（默认）

基本操作

  1. 文件传输
    push

    1
    2
    3
    4
    5
    6
    7
    8
    9
    # push to a folder
    d.push("foo.txt", "/sdcard/")
    # push and rename
    d.push("foo.txt", "/sdcard/bar.txt")
    # push fileobj
    with open("foo.txt", 'rb') as f:
        d.push(f, "/sdcard/")
    # push and change file access mode
    d.push("foo.sh", "/data/local/tmp/", mode=0o755)

    pull

    1
    2
    3
    4
    d.pull("/sdcard/tmp.txt", "tmp.txt")

    # FileNotFoundError will raise if the file is not found on the device
    d.pull("/sdcard/some-file-not-exists.txt", "tmp.txt")

  2. shell

    1
    2
    3
    4
    5
    6
    7
    8
    output, exit_code = d.shell("pwd", timeout=60) # timeout 60s (Default)
    # output: "/\n", exit_code: 0
    # Similar to command: adb shell pwd

    # Since `shell` function return type is `namedtuple("ShellResponse", ("output", "exit_code"))`
    # so we can do some tricks
    output = d.shell("pwd").output
    exit_code = d.shell("pwd").exit_code
1
2
3
4
5
6
7
8
9
10
r = d.shell("logcat", stream=True)
# r: requests.models.Response
deadline = time.time() + 10 # run maxium 10s
try:
    for line in r.iter_lines(): # r.iter_lines(chunk_size=512, decode_unicode=None, delimiter=None)
        if time.time() > deadline:
            break
        print("Read:", line.decode('utf-8'))
finally:
    r.close() # this method must be called

  1. session
    代表一个 app 的生命周期，可以用来启动、停用 app，检测 app crash

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    sess = d.session("com.netease.cloudmusic") # start 网易云音乐
    sess.close() # 停止网易云音乐
    sess.restart() # 冷启动网易云音乐

    with d.session("com.netease.cloudmusic") as sess:
        sess(text="Play").click()

    # launch app if not running, skip launch if already running
    sess = d.session("com.netease.cloudmusic", attach=True)

    # raise SessionBrokenError if not running
    sess = d.session("com.netease.cloudmusic", attach=True, strict=True)

    # When app is still running
    sess(text="Music").click() # operation goes normal

    # If app crash or quit
    sess(text="Music").click() # raise SessionBrokenError
    # other function calls under session will raise SessionBrokenError too

    # check if session is ok.
    # Warning: function name may change in the future
    sess.running() # True or False

  2. 设备信息

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    d.info
    # { 
    #    u'displayRotation': 0,
    #    u'displaySizeDpY': 640,
    #    u'displaySizeDpX': 360,
    #    u'currentPackageName': u'com.android.launcher',
    #    u'productName': u'takju',
    #    u'displayWidth': 720,
    #    u'sdkInt': 18,
    #    u'displayHeight': 1184,
    #    u'naturalOrientation': True
    #}

    print(d.window_size())
    # device upright output example: (1080, 1920)
    # device horizontal output example: (1920, 1080)

    print(d.app_current())
    # Output example 1: {'activity': '.Client', 'package': 'com.netease.example', 'pid': 23710}
    # Output example 2: {'activity': '.Client', 'package': 'com.netease.example'}
    # Output example 3: {'activity': None, 'package': None}

    d.wait_activity(".ApiDemos", timeout=10) # default timeout 10.0 seconds
    # Output: true of false

    print(d.serial)
    # output example: 74aAEDR428Z9

    print(d.wlan_ip)
    # output example: 10.0.0.1

    print(d.device_info)
    # {'udid': '3578298f-b4:0b:44:e6:1f:90-OD103',
    #  'version': '7.1.1',
    #  'serial': '3578298f',
    #  'brand': 'SMARTISAN',
    #  'model': 'OD103',
    #  'hwaddr': 'b4:0b:44:e6:1f:90',
    #  'port': 7912,
    #  'sdk': 25,
    #  'agentVersion': 'dev',
    #  'display': {'width': 1080, 'height': 1920},
    #  'battery': {'acPowered': False,
    #   'usbPowered': False,
    #   'wirelessPowered': False,
    #   'status': 3,
    #   'health': 0,
    #   'present': True,
    #   'level': 99,
    #   'scale': 100,
    #   'voltage': 4316,
    #   'temperature': 272,
    #   'technology': 'Li-ion'},
    #  'memory': {'total': 3690280, 'around': '4 GB'},
    #  'cpu': {'cores': 8, 'hardware': 'Qualcomm Technologies, Inc MSM8953Pro'},
    #  'presenceChangedAt': '0001-01-01T00:00:00Z',
    #  'usingBeganAt': '0001-01-01T00:00:00Z'
    # }

  3. 剪切板

    1
    2
    d.set_clipboard('text', 'label')
    print(d.clipboard)

  4. 事件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    d.screen_on() # turn on the screen
    d.screen_off() # turn off the screen

    d.info.get('screenOn') # require Android >= 4.4

    d.press("home") # press the home key, with key name
    d.press("back") # press the back key, with key name
    d.press(0x07, 0x02) # press keycode 0x07('0') with META ALT(0x02)

    # These key names are currently supported:
    # home
    # back
    # left
    # right
    # up
    # down
    # center
    # menu
    # search
    # enter
    # delete ( or del)
    # recent (recent apps)
    # volume_up
    # volume_down
    # volume_mute
    # camera
    # power

  • 手势操作

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    d.unlock()
    # This is equivalent to
    # 1. launch activity: com.github.uiautomator.ACTION_IDENTIFY
    # 2. press the "home" key

    d.click(x, y)

    d.double_click(x, y)
    d.double_click(x, y, 0.1) # default duration between two click is 0.1s

    d.long_click(x, y)
    d.long_click(x, y, 0.5) # long click 0.5s (default)

    d.swipe(sx, sy, ex, ey)
    d.swipe(sx, sy, ex, ey, 0.5) # swipe for 0.5s(default)

    d.swipe_ext("right") # 手指右滑，4选1 "left", "right", "up", "down"
    d.swipe_ext("right", scale=0.9) # 默认0.9, 滑动距离为屏幕宽度的90%
    d.swipe_ext("right", box=(0, 0, 100, 100)) # 在 (0,0) -> (100, 100) 这个区域做滑动

    # 实践发现上滑或下滑的时候，从中点开始滑动成功率会高一些
    d.swipe_ext("up", scale=0.8) # 代码会vkk
    # 还可以使用Direction作为参数
    from uiautomator2 import Direction
    d.swipe_ext(Direction.FORWARD) # 页面下翻, 等价于 d.swipe_ext("up"), 只是更好理解
    d.swipe_ext(Direction.BACKWARD) # 页面上翻
    d.swipe_ext(Direction.HORIZ_FORWARD) # 页面水平右翻
    d.swipe_ext(Direction.HORIZ_BACKWARD) # 页面水平左翻

    d.drag(sx, sy, ex, ey)
    d.drag(sx, sy, ex, ey, 0.5) # swipe for 0.5s(default)

    # swipe from point(x0, y0) to point(x1, y1) then to point(x2, y2)
    # time will speed 0.2s bwtween two points
    d.swipe_points([(x0, y0), (x1, y1), (x2, y2)], 0.2)

    d.touch.down(10, 10) # 模拟按下
    time.sleep(.01) # down 和 move 之间的延迟，自己控制
    d.touch.move(15, 15) # 模拟移动
    d.touch.up() # 模拟抬起

  • 屏幕相关

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    # retrieve orientation. the output could be "natural" or "left" or "right" or "upsidedown"
    orientation = d.orientation
    # WARNING: not pass testing in my TT-M1
    # set orientation and freeze rotation.
    # notes: setting "upsidedown" requires Android>=4.3.
    d.set_orientation('l') # or "left"
    d.set_orientation("l") # or "left"
    d.set_orientation("r") # or "right"
    d.set_orientation("n") # or "natural"

    # freeze rotation
    d.freeze_rotation()
    # un-freeze rotation
    d.freeze_rotation(False)

    # take screenshot and save to a file on the computer, require Android>=4.2.
    d.screenshot("home.jpg")
    # get PIL.Image formatted images. Naturally, you need pillow installed first
    image = d.screenshot() # default format="pillow"
    image.save("home.jpg") # or home.png. Currently, only png and jpg are supported
    # get opencv formatted images. Naturally, you need numpy and cv2 installed first
    import cv2
    image = d.screenshot(format='opencv')
    cv2.imwrite('home.jpg', image)
    # get raw jpeg data
    imagebin = d.screenshot(format='raw')
    open("some.jpg", "wb").write(imagebin)

    # get the UI hierarchy dump content (unicoded).
    xml = d.dump_hierarchy()

    d.open_notification()
    d.open_quick_settings()

  • 选择器（selector）

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    # Select the object with text 'Clock' and its className is 'android.widget.TextView'
    d(text='Clock', className='android.widget.TextView')

    # Selector supports below parameters. Refer to UiSelector Java doc for detailed information.
    # text, textContains, textMatches, textStartsWith
    # className, classNameMatches
    # description, descriptionContains, descriptionMatches, descriptionStartsWith
    # checkable, checked, clickable, longClickable
    # scrollable, enabled,focusable, focused, selected
    # packageName, packageNameMatches
    # resourceId, resourceIdMatches
    # index, instance

    # get the children or grandchildren
    d(className="android.widget.ListView").child(text="Bluetooth")

    # get siblings
    d(text="Google").sibling(className="android.widget.ImageView")

    # get the child matching the condition className="android.widget.LinearLayout"
    # and also its children or grandchildren with text "Bluetooth"
    d(className="android.widget.ListView", resourceId="android:id/list") \
     .child_by_text("Bluetooth", className="android.widget.LinearLayout")
    # get children by allowing scroll search
    d(className="android.widget.ListView", resourceId="android:id/list") \
     .child_by_text(
        "Bluetooth",
        allow_scroll_search=True,
        className="android.widget.LinearLayout"
      )

    ## select "switch" on the right side of "Wi‑Fi"
    d(text="Wi‑Fi").right(className="android.widget.Switch").click()
    d(text="Add new", instance=0)  # which means the first instance with text "Add new"

    # get the count of views with text "Add new" on current screen
    d(text="Add new").count
    # same as count property
    len(d(text="Add new"))
    # get the instance via index
    d(text="Add new")[0]
    d(text="Add new")[1]
    ...
    # iterator
    for view in d(text="Add new"):
        view.info  # ...

获取到目标元素后查看状态与信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
d(text="Settings").exists # True if exists, else False
d.exists(text="Settings") # alias of above property.

# advanced usage
d(text="Settings").exists(timeout=3) # wait Settings appear in 3s, same as .wait(3)

d(text="Settings").info

d(text="Settings").get_text()  # get widget text
d(text="Settings").set_text("My text...")  # set the text
d(text="Settings").clear_text()  # clear the text

x, y = d(text="Settings").center()
# x, y = d(text="Settings").center(offset=(0, 0)) # left-top x, y

im = d(text="Settings").screenshot()
im.save("settings.jpg")

# click on the center of the specific ui object
d(text="Settings").click()

# wait element to appear for at most 10 seconds and then click
d(text="Settings").click(timeout=10)

# click with offset(x_offset, y_offset)
# click_x = x_offset * width + x_left_top
# click_y = y_offset * height + y_left_top
d(text="Settings").click(offset=(0.5, 0.5)) # Default center
d(text="Settings").click(offset=(0, 0)) # click left-top
d(text="Settings").click(offset=(1, 1)) # click right-bottom

# click when exists in 10s, default timeout 0s
clicked = d(text='Skip').click_exists(timeout=10.0)

# click until element gone, return bool
is_gone = d(text="Skip").click_gone(maxretry=10, interval=1.0) # maxretry default 10, interval default 1.0

# long click on the center of the specific UI object
d(text="Settings").long_click()

# notes : drag can not be used for Android<4.3.
# drag the UI object to a screen point (x, y), in 0.5 second
d(text="Settings").drag_to(x, y, duration=0.5)
# drag the UI object to (the center position of) another UI object, in 0.25 second
d(text="Settings").drag_to(text="Clock", duration=0.25)

d(text="Settings").swipe("right")
d(text="Settings").swipe("left", steps=10)
d(text="Settings").swipe("up", steps=20) # 1 steps is about 5ms, so 20 steps is about 0.1s
d(text="Settings").swipe("down", steps=20)

d(text="Settings").gesture((sx1, sy1), (sx2, sy2), (ex1, ey1), (ex2, ey2))

# notes : pinch can not be set until Android 4.3.
# from edge to center. here is "In" not "in"
d(text="Settings").pinch_in(percent=100, steps=10)
# from center to edge
d(text="Settings").pinch_out()

# wait until the ui object appears
d(text="Settings").wait(timeout=3.0) # return bool
# wait until the ui object gone
d(text="Settings").wait_gone(timeout=1.0)

# fling forward(default) vertically(default) 
d(scrollable=True).fling()
# fling forward horizontally
d(scrollable=True).fling.horiz.forward()
# fling backward vertically
d(scrollable=True).fling.vert.backward()
# fling to beginning horizontally
d(scrollable=True).fling.horiz.toBeginning(max_swipes=1000)
# fling to end vertically
d(scrollable=True).fling.toEnd()

# scroll forward(default) vertically(default)
d(scrollable=True).scroll(steps=10)
# scroll forward horizontally
d(scrollable=True).scroll.horiz.forward(steps=100)
# scroll backward vertically
d(scrollable=True).scroll.vert.backward()
# scroll to beginning horizontally
d(scrollable=True).scroll.horiz.toBeginning(steps=100, max_swipes=1000)
# scroll to end vertically
d(scrollable=True).scroll.toEnd()
# scroll forward vertically until specific ui object appears
d(scrollable=True).scroll.to(text="Security")
  1. WatchContext
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    with d.watch_context() as ctx:
        ctx.when("^立即(下载|更新)").when("取消").click() # 当同时出现 （立即安装 或 立即取消）和 取消 按钮的时候，点击取消
        ctx.when("同意").click()
        ctx.when("确定").click()
        # 上面三行代码是立即执行完的，不会有什么等待
        
        ctx.wait_stable() # 开启弹窗监控，并等待界面稳定（两个弹窗检查周期内没有弹窗代表稳定）

        # 使用call函数来触发函数回调
        # call 支持两个参数，d和el，不区分参数位置，可以不传参，如果传参变量名不能写错
        # eg: 当有元素匹配仲夏之夜，点击返回按钮
        ctx.when("仲夏之夜").call(lambda d: d.press("back"))
        ctx.when("确定").call(lambda el: el.click())

        # 其他操作

    # 为了方便也可以使用代码中默认的弹窗监控逻辑
    # 下面是目前内置的默认逻辑，可以加群at群主，增加新的逻辑，或者直接提pr
        # when("继续使用").click()
        # when("移入管控").when("取消").click()
        # when("^立即(下载|更新)").when("取消").click()
        # when("同意").click()
        # when("^(好的|确定)").click()
    with d.watch_context(builtin=True) as ctx:
        # 在已有的基础上增加
        ctx.when("@tb:id/jview_view").when('//*[@content-desc="图片"]').click()

        # 其他脚本逻辑
1
2
3
4
5
ctx = d.watch_context()
ctx.when("设置").click()
ctx.wait_stable() # 等待界面不在有弹窗了

ctx.close()

  1. 全局设置

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    d.HTTP_TIMEOUT = 60 # 默认值60s, http默认请求超时时间

    # 当设备掉线时，等待设备在线时长，仅当TMQ=true时有效，支持通过环境变量 WAIT_FOR_DEVICE_TIMEOUT 设置
    d.WAIT_FOR_DEVICE_TIMEOUT = 70 

    print(d.settings)
    {'operation_delay': (0, 0),
     'operation_delay_methods': ['click', 'swipe'],
     'wait_timeout': 20.0,
     'xpath_debug': False}

    # 配置点击前延时0.5s，点击后延时1s
    d.settings['operation_delay'] = (.5, 1)

    # 修改延迟生效的方法
    # 其中 double_click, long_click 都对应click
    d.settings['operation_delay_methods'] = ['click', 'swipe', 'drag', 'press']

    d.settings['xpath_debug'] = True # 开启xpath插件的调试日志
    d.settings['wait_timeout'] = 20.0 # 默认控件等待时间（原生操作，xpath插件的等待时间）

    >>> d.settings['click_before_delay'] = 1  
    [W 200514 14:55:59 settings:72] d.settings[click_before_delay] deprecated: Use operation_delay instead

  2. 输入

    1
    2
    3
    4
    5
    d.set_fastinput_ime(True) # 切换成FastInputIME输入法
    d.send_keys("你好123abcEFG") # adb广播输入
    d.clear_text() # 清除输入框所有内容(Require android-uiautomator.apk version >= 1.0.7)
    d.set_fastinput_ime(False) # 切换成正常的输入法
    d.send_action("search") # 模拟输入法的搜索

  3. Toast

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    d.toast.show("Hello world")
    d.toast.show("Hello world", 1.0) # show for 1.0s, default 1.0s

    # [Args]
    # 5.0: max wait timeout. Default 10.0
    # 10.0: cache time. return cache toast if already toast already show up in recent 10 seconds. Default 10.0 (Maybe change in the furture)
    # "default message": return if no toast finally get. Default None
    d.toast.get_message(5.0, 10.0, "default message")
    # common usage
    assert "Short message" in d.toast.get_message(5.0, default="")
    # clear cached toast
    d.toast.reset()
    # Now d.toast.get_message(0) is None

  4. XPath

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    # wait exists 10s
    d.xpath("//android.widget.TextView").wait(10.0)
    # find and click
    d.xpath("//*[@content-desc='分享']").click()
    # check exists
    if d.xpath("//android.widget.TextView[contains(@text, 'Se')]").exists:
        print("exists")
    # get all text-view text, attrib and center point
    for elem in d.xpath("//android.widget.TextView").all():
        print("Text:", elem.text)
        # Dictionary eg: 
        # {'index': '1', 'text': '999+', 'resource-id': 'com.netease.cloudmusic:id/qb', 'package': 'com.netease.cloudmusic', 'content-desc': '', 'checkable': 'false', 'checked': 'false', 'clickable': 'false', 'enabled': 'true', 'focusable': 'false', 'focused': 'false','scrollable': 'false', 'long-clickable': 'false', 'password': 'false', 'selected': 'false', 'visible-to-user': 'true', 'bounds': '[661,1444][718,1478]'}
        print("Attrib:", elem.attrib)
        # Coordinate eg: (100, 200)
        print("Position:", elem.center())

  5. Screenrecord

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    d.screenrecord('output.mp4')

    time.sleep(10)
    # or do something else
    d.screenrecord.stop() # 停止录制后，output.mp4文件才能打开

    imdata = "target.png" # 也可以是URL, PIL.Image或OpenCV打开的图像

    d.image.match(imdata) 
    # 匹配待查找的图片，立刻返回一个结果
    # 返回一个dict, eg: {"similarity": 0.9, "point": [200, 300]}

    d.image.click(imdata, timeout=20.0)
    # 在20s的时间内调用match轮询查找图片，当similarity>0.9时，执行点击操作