Automated capture of application user interfaces

Can’t you drive the app and capture most of the screens needed for visual context?


Once all the strings that need to be translated have been identified, the developer then needs to go find those strings in the app interfaces so that the visual context for each translation can be captured. Although using the app to find strings doesn’t have to take much time, the thought was, why can’t we drive the app and find many of the strings automatically to save the developer even more time?


Each app screen has user interface elements; buttons, menus, sliders, etc. Frequently these interface elements cause the display to change or they cause the app to navigate to another screen. We can automate many user interface activities and quickly drive the app to a large portion of the screens. Essentially crawl the app similar to how a web search engine crawls a web site. Follow all the buttons.

This should capture a significant portion of the screens / strings in the app. The developer would then look at the remaining unfound strings and manually navigate through the app interfaces to capture those remaining screens / strings.

This technique typically does not capture all screens because some require user data to cause them to appear. For example, perhaps entering a bad email address causes an alert. This system wouldn’t enter bad emails to cause that alert. The developer would do that.


Makes the first gathering of all the screens / strings for visual context much faster, saving the developer time. For a catalog type app, with hundreds of screens displaying product information, the time savings can be enormous.


US Pattent Application Number 62152968
Filed 26-APR-2016