Seite - 41 - in Austrian Law Journal, Band 1/2019

ALJ 2019 Digital Single Market – towards Smart Regulations 41 the latter. Currently, AI is already implemented in e-government, e-business, e-commerce/e- shopping, e-library services, e-learning, e-tourism, e-resource services and e-group activities. Typically, machine-learning recommenders are designed to detect our preferences and to recommend news, products, webpages, trips etc.20 These recommendations can result in echo chambers helping to spread misinformation as, for example, in the case of the BREXIT referendum.21 Algorithms have already been used in the public sector for some time. Only recently, however, an algorithmic system has been the topic of heated discussions in the Austrian media. Since the beginning of 2019, the Austrian Public Employment Service (AMS) has been using a machine- learning algorithm to divide job seekers into three categories based on general criteria without taking the individual circumstances of the job-seeker into account. This was criticized as a potential threat to certain groups: For example, women are generally rated more negatively in the evaluation of their job opportunities solely on the basis of their gender. The same applies to other structurally disadvantaged groups such as migrants and people with disabilities.22 “Predictive policing” is another example for the use of algorithms by state authorities:23 The Austrian police uses algorithms to define zones in which there is an increased risk of burglary, thus preventing crimes of this kind more effectively.24 The use of machine-learning algorithms, in particular by the state, raises numerous fundamental legal questions. Regarding legislation, the question arises how accurately their use has to be regulated. As to the use of self-learning algorithms by the executive, the main questions focus on the limits of its admissibility. The assistance provided by private actors in the use of algorithms by the state, e.g. by training or by providing information to it,25 raises further questions. So far, neither EU law nor domestic law offer comprehensive answers to these and many other legal questions arising from the use of AI. At the European level, however, efforts are already being 20 Cf. Lu, Wu, Mao, Wang and Zhang, Recommender system application developments: a survey, 74 DECISION SUPPORT SYSTEMS 12 (2015), available at https://opus.lib.uts.edu.au/handle/10453/35614. 21 Del Vicario, Zollo, Caldarelli, Scala and Quattrociocchi, Mapping social dynamics on Facebook: The Brexit debate, 50 SOCIAL NETWORKS 6 (2017), available at https://www.sciencedirect.com/science/article/pii/S0378873316304166?via%3Dihub. 22 Cf. Stuiber, Volksanwaltschaft prüft AMS-Algorithmus, DER STANDARD, Nov. 2, 2018, available at https://derstandard.at/2000090540950/Volksanwaltschaft-prueft-AMS-Algorithmus. Concerning possible ways to handle “Machine Bias“ cf. von Lewinski, supra note 11, at 173. 23 Cf. Knobloch, Vor die Lage kommen: Predictive Policing in Deutschland (2018) Bertelsmann Stiftung https://www.bertelsmann- stiftung.de/fileadmin/files/BSt/Publikationen/GrauePublikationen/predictive.policing.pdf; McCarthy, AI & Global Governance: Turning the Tide on Crime with Predictive Policing, CENTRE FOR POLICY RESEARCH, UNITED NATIONS UNIVERSITY, Feb. 26, 2019, available at https://cpr.unu.edu/ai-global-governance-turning-the-tide-on-crime-with- predictive-policing.html; Selbst, Disparate Impact in Big Data Policing, 52 GEORGIA LAW REVIEW 109 (2017). 24 Cf. Al-Youssef and Sulzbacher, Polizei nutzt Algorithmen, um Verbrechen vorherzusehen – Kritiker sehen "Aberglaube", DER STANDARD, Nov. 3, 2018, available at https://derstandard.at/2000090545361/Polizei-nutzt- Algorithmen-um-Verbrechen-vorherzusehen-Kritiker-sehen-Aberglaube; van der Aalst, Bichler and Heinzl, Responsible Data Science, 59 BUSINESS AND INFORMATION SYSTEMS ENGINEERING 311 (2017); van der Aalst, Responsible Data Science: Using Big Data in a "People Friendly" Manner, in ENTERPRISE INFORMATION SYSTEMS 3 (Hammoudi, Maciaszek, Missikoff, Camp and Cordiero ed., 2017). 25 This question is closely related to the one concerning the legitimation of state acts (cf. Klafki, Würkert and Winter, Digitalisierung und Öffentliches Recht, in DIGITALISIERUNG UND RECHT 18 (Klafki, Würkert and Winter ed., 2017).