Help through Demonstration and Automation for Interactive Computing Systems: a Survey of RecentWorks

Jose Silva

Abstract


Usability is very important however, it is still difficult to develop interactive computing systems that meet all user’s specificities. Help systems should be a way of bridging this gap. This paper presents a general survey on recent works (building upon previous surveys) related to improving applications help through demonstration and automation and, identifies which technologies are acting as enablers. The main contributions are, identifying (1) which are the recent existing solutions, (2) which aspects must be investigated further and (3) which are the main difficulties that are preventing a faster progress.

Keywords


Help system; Demonstration-based help; Automation; Picture-driven computing; Learning

References


Norman, D. A. (1998). The invisible computer: why good products can fail, the personal computer is so complex, and information appliances are the solution. MIT press.

Akiki PA (2018) CHAIN: Developing model-driven contextual help for adaptive user interfaces. Journal of Systems and Software, 135, 165-190.

Akiki PA, Bandara AK, Yu Y (2013, June) Cedar studio: an IDE supporting adaptive model-driven user interfaces for enterprise applications. In Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems (pp. 139-144). ACM.

Allen J, Chambers N, Ferguson G, Galescu L, Jung H, Swift M, Taysom W (2007, July) Plow: A collaborative task learning agent. In AAAI (Vol. 7, pp. 1514-1519).

Amershi S, Mahmud J, Nichols J, Lau T, Ruiz GA (2013) LiveAction: Automating web task model generation. ACM Transactions on Interactive Intelligent Systems (TiiS), 3(3), 14.

Bolognesi T, Brinksma E (1987) Introduction to the ISO specification language LOTOS. Computer Networks and ISDN systems, 14(1), 25-59.

Boy GA (1998, January) Cognitive function analysis for human-centered automation of safety-critical systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 265-272). ACM Press/Addison-Wesley Publishing Co..

Card SK (2017) The psychology of human-computer interaction. CRC Press.

Chilana PK, Ko AJ, Wobbrock JO (2012, May) LemonAid: selection-based crowdsourced contextual help for web applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1549-1558). ACM.

Chilana PK, Ko AJ, Wobbrock JO, Grossman T (2013, April) A multi-site field study of crowdsourced contextual help: usage and perspectives of end users and software teams. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 217-226). ACM.

Dixon M, Fogarty J, Wobbrock J (2012, May) A general-purpose target-aware pointing enhancement using pixel-level analysis of graphical interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 3167-3176). ACM.

Dixon M, Nied A, Fogarty J (2014, October) Prefab layers and prefab annotations: extensible pixel-based interpretation of graphical interfaces. In Proceedings of the 27th annual ACM symposium on User interface software and technology (pp. 221-230). ACM.

Dragunov AN, Dietterich TG, Johnsrude K, McLaughlin M, Li L, Herlocker JL (2005, January). TaskTracer: a desktop environment to support multi-tasking knowledge workers. In Proceedings of the 10th international conference on Intelligent user interfaces (pp. 75-82). ACM.

Dworman G (2007) Arbitration of a help system. interactions, 14(1), 39-42.

Dworman G, Rosenbaum S (2004, April) Helping users to use help: improving interaction with help systems. In CHI'04 extended abstracts on Human factors in computing systems (pp. 1717-1718). ACM.

Grossman T, Fitzmaurice G (2010, April) ToolClips: an investigation of contextual video assistance for functionality understanding. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1515-1524). ACM.

iCS, HAMSTERS, Inst. Rechearch En Inform. Toulouse. https://www.irit.fr/recherches/ICS/softwares/ hamsters/index.html, Accessed 14 January 2020

Intharah T, Turmukhambetov D, Brostow GJ (2017, March) Help, It Looks Confusing: GUI Task Automation Through Demonstration and Follow-up Questions. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (pp. 233-243). ACM.

Kourousias G, Bonfiglio S (2010, October) Picture-Driven Computing In Assistive Technology And Accessibility Design. In 1st International ÆGIS Conference (Vol. 7, p. 8).

Lau T, Bergman L, Castelli V, Oblinger D (2004, January) Sheepdog: learning procedures for technical support. In Proceedings of the 9th international conference on Intelligent user interfaces (pp. 109-116). ACM.

Lau T, Cerruti J, Manzato G, Bengualid M, Bigham JP, Nichols J (2010, October) A conversational interface to web automation. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (pp. 229-238). ACM.

Lazar J, Jones A, Shneiderman B (2006) Workplace user frustration with computers: An exploratory investigation of the causes and severity. Behaviour & Information Technology, 25(03), 239-251.

Leshed G, Haber EM, Matthews T, Lau T (2008, April) CoScripter: automating & sharing how-to knowledge in the enterprise. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1719-1728). ACM.

Li I, Nichols J, Lau T, Drews C, Cypher A (2010, April) Here's what i did: sharing and reusing web activity with ActionShot. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 723-732). ACM.

Machado V, Lopes N, Silva JC, Silva JL (2017, April) Picture-Based Task Definition and Parameterization Support System. In World Conference on Information Systems and Technologies (pp. 592-601). Springer, Cham.

Martinie C, Palanque P, Barboni E, Ragosta M (2011, October) Task-model based assessment of automation levels: application to space ground segments. In Systems, Man, and Cybernetics (SMC), 2011 IEEE International Conference on (pp. 3267-3273). IEEE.

Martinie C, Palanque P, Navarre D, Winckler M, Poupart E (2011, June) Model-based training: an approach supporting operability of critical interactive systems. In Proceedings of the 3rd ACM SIGCHI symposium on Engineering interactive computing systems (pp. 53-62). ACM.

Matejka J, Grossman T, Fitzmaurice G (2011, May) Ambient help. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2751-2760). ACM.

Murata T (1989) Petri nets: Properties, analysis and applications. Proceedings of the IEEE, 77(4), 541-580.

Navarre D, Palanque P, Ladry JF, Barboni E (2009) ICOs: A model-based user interface description technique dedicated to interactive systems addressing usability, reliability and scalability. ACM Transactions on Computer-Human Interaction (TOCHI), 16(4), 18.

Ornelas JD, Silva JC, Silva JL (2016, April) Demonstration-based Help for Interactive Systems. In Proceedings of the 2nd International Conference in HCI and UX Indonesia 2016 (pp. 125-128). ACM.

Paternò F (2004). ConcurTaskTrees: an engineered notation for task models. The handbook of task analysis for human-computer interaction, 483-503.

Paternò F, Mancini C, Meniconi S (1997) ConcurTaskTrees: A diagrammatic notation for specifying task models. In Human-Computer Interaction INTERACT’97 (pp. 362-369). Springer, Boston, MA.

Pongnumkul S, Dontcheva M, Li W, Wang J, Bourdev L, Avidan S, Cohen MF (2011, October) Pause-and-play: automatically linking screencast video tutorials with applications. In Proceedings of the 24th annual ACM symposium on User interface software and technology (pp. 135-144). ACM.

Rodrigues P, Silva JL, Pereira R (2018) DEMONSTRATION-BASED HELP: A CASE STUDY, 10th annual International Conference on Education and New Learning Technologies (in press).

Rosenbaum S, Ramey J, Redish JG (2010, April) Current issues in assessing and improving information usability. In CHI'10 Extended Abstracts on Human Factors in Computing Systems (pp. 3155-3158). ACM.

Schatsky D, Muraskin C, Iyengar K, Robotic process automation: A path to the cognitive enterprise, Deloitte, https://www2.deloitte.com/insights/us/en/focus/signals-for-strategists/cognitive-enterprise-robotic-process-automation.html, Accessed 14 January 2020

Shi, JJ, Lee DE, Kuruku E (2008) Task-based modeling method for construction business process modeling and automation. Automation in Construction, 17(5), 633-640.

Silva JL, Ornelas JD, Silva JC (2016, June) Make it ISI: interactive systems integration tool. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (pp. 245-250). ACM.

Stohr EA, Zhao JL (2001) Workflow automation: Overview and research issues. Information Systems Frontiers, 3(3), 281-296.

Sun Y, Chen D, Xin C, Jiao W (2015, July) Automating Repetitive Tasks on Web-based IDEs via an Editable and Reusable Capture-Replay Technique. In Computer Software and Applications Conference (COMPSAC), 2015 IEEE 39th Annual (Vol. 2, pp. 666-675). IEEE. IEEE, 666, 2015

Sun Y, QiaO Z, Chen D, Xin C, Jiao W (2016, June). An Approach to Using Existing Online Education Tools to Support Practical Education on MOOCs. In Computer Software and Applications Conference (COMPSAC), 2016 IEEE 40th Annual (Vol. 1, pp. 696-705). IEEE.

Utermohlen K All the Robotic Process Automation (RPA) Stats You Need to Know, Data Sci. https://towardsdatascience.com/all-the-robotic-process-automation-rpa-stats-you-need-to-know-bcec22eaaad9, Accessed 14 January 2020

Vogel-Heuser B, Diedrich C, Fay A, Jeschke S, Kowalewski S, Wollschlaeger M, Göhner P (2014) Challenges for software engineering in automation. Journal of Software Engineering and Applications, 7(05), 440.

Vouligny L, Robert JM (2005, October) Online help system design based on the situated action theory. In Proceedings of the 2005 Latin American conference on Human-computer interaction (pp. 64-75). ACM.

Winter J, Rönkkö K, Rissanen M (2014) Identifying organizational barriers—A case study of usability work when developing software in the automation industry. Journal of Systems and Software, 88, 54-73.

Yeh T, Chang TH, Miller RC (2009, October) Sikuli: using GUI screenshots for search and automation. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (pp. 183-192). ACM.

Yeh T, Chang TH, Xie B, Walsh G, Watkins I, Wongsuphasawat K, Huang M, Davis LS, Bederson BB (2011, October) Creating contextual help for GUIs using screenshots. In Proceedings of the 24th annual ACM symposium on User interface software and technology (pp. 145-154). ACM.

Instructional Systems Development (ISD)., Air Univ, http://www.au.af.mil/au/awc/awcgate/doe/isd/paper.htm, Accessed 14 January 2020

Grossman, T., & Fitzmaurice, G. (2010, April). ToolClips: an investigation of contextual video assistance for functionality understanding. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1515-1524). ACM.

Dubrovina, A., Kisilev, P., Freedman, D., Schein, S., & Bergman, R. (2012, November). Efficient and robust image descriptor for GUI object classification. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012) (pp. 3594-3597). IEEE.

Freedman, D., Kisilev, P., Dubrovina, A., Schein, S., & Bergman, R. (2015). U.S. Patent No. 9,213,463. Washington, DC: U.S. Patent and Trademark Office.

https://www.automationanywhere.com, Accessed 14 January 2020

https://www.blueprism.com, Accessed 14 January 2020

http://hiis.isti.cnr.it:4500/home, Accessed 14 January 2020

https://www.nice.com, Accessed 14 January 2020

https://www.uipath.com, Accessed 14 January 2020




DOI: http://doi.org/10.11591/ijece.v11i2.pp%25p
Total views : 0 times


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

ISSN 2088-8708, e-ISSN 2722-2578