Mass data collection, often by means that are not obvious to individuals; vague or misleading collection notices; and an assumption that people are more comfortable with the secondary use of their information than they actually are, lead to a situation in which the current understanding of information privacy through these principles may no longer be effective. The assumption that people, particularly young people or ‘digital natives’, are becoming less concerned about their information privacy may prompt the idea that a reasonably expected secondary purpose for use of information would be quite broad. Combining this with the issues of purpose specification above, organisations are likely to find it difficult to ensure personal information is only used for the purpose it was collected for when using AI technologies. Just as AI can highlight patterns and relationships in data unforeseen by humans, it could also reveal new potential uses for that information. In general, organisations are also permitted to use personal information for a secondary purpose that would be ‘reasonably expected’ by the individual. In this way AI could be pivotal in the establishment of individualised, preference-based models that have the potential to meet the transparency, consent and reasonable expectations objectives of information privacy law, even more effectively than the current model of notice and consent.
As with other areas of data-intensive technology application, there are problems with the enforcement of data subjects’ rights in the case of generative models (Solove, Reference Solove2023). For instance, the research exemption under article 9(2)(j), for instance, is restricted to the development of models for research purposes and does not permit their commercial exploitation, as indicated in Recitals 159 and 162 (Novelli et al., Reference Novelli, Casolari, Hacker, Spedicato and Floridi2024). Consequently, in many instances involving big data, merely being able to potentially infer sensitive information may subject processes such as AI training to the provisions of article 9, and there is little likelihood that LLMs satisfy the exceptions in article 9(2). In addition, generative models are scalable in terms of their output, which means that false information can be disseminated to a large number of users and third parties. Theoretically, legitimate interest could also be considered here under article 6(1)(f), but must be assessed on a case-by-case basis according to the criteria described above.
Collecting data only for lawful, specific purposes aligns with both data subject expectations and regulatory requirements across jurisdictions. Assessments should address data flows, algorithmic outputs, and potential privacy impacts as three distinct areas of scrutiny. Risk assessments for AI must evaluate not just direct data collection but also the potential for systems to infer sensitive information from benign inputs. China’s Interim Measures for Administration of Generative AI Services (2023) protect personal information and privacy rights, prohibiting AI applications that harm mental or physical health or infringe on individual reputation or privacy. California leads with the California Consumer Privacy Act (CCPA) and the California Privacy Rights Act (CPRA), which establish transparency requirements and consumer rights regarding the use of personal data in AI applications. The Act prohibits certain AI applications entirely, including social scoring systems and real-time biometric identification in public spaces.
- Despite these challenges, it is important that organisations take steps to ensure they are transparent about their handling of personal information in relation to AI systems.
- As AI technology continues to advance, it raises important ethical questions about the use of personal data and the potential for bias in AI systems.
- International standards and tools can aid with regulatory compliance and enforcement.
- How can we prevent the potential harms and biases that may result from the use of AI systems?
European Ethical Charter on the use of artificial intelligence in… (
Throughout the lifecycle of the AI product, your organisation should have in place processes for ensuring that the product continues to be reliable and appropriate for its intended uses. It is critical to ensure that you can provide the client with a sufficient explanation about how the decision was reached and the role that the AI product played in this process. To ensure your use of AI is transparent to your clients, you should ensure that the use of personal information for these purposes is clearly outlined in your Privacy Policy.
Current browse context:
- Because AI systems rely on the data they process for functionality and value, it is essential to ensure that this data remains confidential, accurate, and available throughout the lifecycle.
- These risks can be compounded by the tendency of generative AI tools to confidently produce outputs which appear credible, regardless of their accuracy.
- To keep defenses effective, organizations should set up a regular peer review cycle—say, every 6 to 12 months—to reassess safeguards against new attack methods.
- Generative AI also has the potential to emulate human-like behaviours and generate realistic outputs, which may cause users to overestimate its accuracy and reliability.
- Some applications, like unjustified mass surveillance, may be banned outright.
Depending on your circumstances, you could base your processing of personal data for both development and ongoing use of AI on the legitimate interests lawful basis. For the development of potentially life-saving AI systems, it would be better to rely on other lawful bases. EDPB guidelines on processing under Article 6(1)(b) in the context of online services. Conversely, use of AI to process personal data for purposes of personalising content may be regarded as necessary for the performance of a contract – but only in some cases. You should also note that you are unlikely to be able to rely on this basis for processing personal data for purposes such as ‘service improvement’ of your AI system. Since machine learning models are typically built using very large datasets, whether or not a single individual’s data is included in the training data should have a negligible effect on the system’s performance.
GDPRLocal’s AI Compliance service helps organisations assess their AI systems against EU AI Act requirements and build the documentation needed for high-risk classification. This acceleration reflects growing recognition of the unique challenges AI poses and its widespread adoption across critical sectors. Traditional data protection measures may not be sufficient against attacks targeting the specific architectures and data flows of modern AI applications. High-risk AI systems containing sensitive training data are targets for cybercriminals seeking to extract valuable personal information. The March 2023 ChatGPT incident, in which users gained access to conversation titles from unrelated accounts, illustrates how technical vulnerabilities in AI models can expose personal information at scale.
- There are three elements to the legitimate interests lawful basis, and it can help to think of these as the ‘three-part test’.
- Staying informed about changes in data protection law and the privacy policies of services you use helps you make better decisions about your digital footprint over time.
- The requirement of ‘specific’ and ‘informed’ consent may also pose significant challenges where the controller neither knows nor is able to foresee how and for which purposes the personal data will be processed by self-learning and autonomous AI systems.
- In this context and in connection with all related questions, the decisive issue is, again, the possibility of a re-identification of the data subject(s).
- “Like any powerful technology, AI does have the potential to be misused in ways that could compromise privacy,” Gilbert says.
- For organisations deploying AI at scale, appointing a Data Protection Officer provides that accountability and ensures a named individual is responsible for compliance.
Scope of the GDPR and legal basis
When employees are blocked from AI tools at work, they use personal accounts on https://www.quickza.com/addressing-cybersecurity-proactively-to-support-hybrid-learning.html personal devices. Blocking AI tools is the instinctive response, and it is understandable. A global aircraft manufacturer that secured more than 7,000 remote employees, contractors, and suppliers found that issuing managed laptops at that scale was neither practical nor cost-effective.
For further support and information, please visit our website
Given the significant and complex privacy risks involved, as a matter of best practice it is recommended that organisations do not enter personal information, and particularly sensitive information, into AI chatbots. If you intend to use personal information in AI systems for other, secondary purposes, you should consider whether these will be authorised by one of the exceptions under APP 6. Your organisation should identify the anticipated purposes for which you will use personal information in connection with an AI system, and whether these are the same as the purposes for which you collected the information. APP 6 provides that an individual’s personal information can only be used or disclosed for the purpose or purposes for which it was collected (known as the ‘primary purpose’) or for a secondary purpose if an exception applies.
Key principles of AI data security
Adversarial training works by stimulating potential attack scenarios for AI to learn and recognize. Using ways such as outlier detection and data cleaning can maintain an approximation to integrity in training datasets, which will act as a fundamental system preventing poisoning attacks. This includes looking at data inputs in detail for irregularities, discrepancies, or potential attack vectors.
US Privacy Regulations
Data from certain domains should be subject to extra protection and used only in “narrowly defined contexts.” These “sensitive domains” include health, employment, education, criminal justice and personal finance. Privacy risks should be assessed and addressed throughout the development lifecycle of an AI system. Under the principle https://thejuon.com/staying-safe-online-new-cybersecurity-measures.html of purpose limitation, companies must have a specific, lawful purpose in mind for any data they collect. For instance, in prompt injection attacks, hackers disguise malicious inputs as legitimate prompts, manipulating generative AI systems into exposing sensitive data.
Threats During Data Processing and Model Training
The National AI Centre has developed a Voluntary AI Safety Standard to help organisations develop and deploy AI systems in Australia safely and reliably. Vulnerable groups, including First Nations people, will often not be properly represented in datasets which reflect historical biases or do not include sufficient data. Organisations should be mindful that the impacts of the use of AI systems may be particularly acute for children and people experiencing vulnerability. The Privacy Act 1988 and the Australian Privacy Principles (APPs) apply to all uses of AI involving personal information, including where information is used to train, test or use an AI system.
Common types of AI tools and products currently being deployed by Australian entities include chatbots, content-generation tools (including text-to-image generators), and productivity assistants that augment writing, coding, note-taking, and transcription. This guidance is intended to assist organisations to comply with their privacy obligations when using commercially available AI products. Deployment can be used for internal purposes or used externally impacting others, such as customers or individuals, who are not deployers of the system. This guidance is targeted at organisations that are deploying AI systems that were built with, collect, store, use or disclose personal information. The role of data protection law and non-discrimination law in group profiling in the private sector. ChatGPT provides false information about people, and OpenAI can’t correct it.