Need for AI regulation in education: risks & standards

The need for AI regulation in education: why schools should slow down before they speed up

A teacher comparing AI feedback tools, a parent reviewing a tutoring app, or a student using a chatbot faces the same basic problem: the product may be easy to access, but its safety and limits may be difficult to judge. The need for AI regulation is not an argument for stopping useful technology. It is an argument for requiring evidence before tools are trusted with student data, classroom decisions, or advice to children.

The pace of change explains the concern. The cost of querying an AI model performing at the level of GPT-3.5 on the MMLU benchmark, a test of broad academic knowledge, fell from $20 per million tokens in November 2022 to $0.07 in October 2024, a reduction of more than 280-fold. Stanford’s 2025 AI Index, published last year, also reported that organizational AI use rose from 55% in 2023 to 78% in 2024, while generative AI use in at least one business function rose from 33% to 71%. (Stanford’s 2025 AI Index)

That combination of lower cost and wider use does not prove that every school is adopting AI. It does show why schools, families, and education companies may encounter these tools before consistent evaluation practices are in place. The practical question is not whether AI belongs in education at all. It is which safeguards should come before a tool influences learning.

Why the need for AI regulation is growing in schools

AI capability is no longer limited to the largest systems or companies. In 2022, the smallest model scoring above 60% on the Massive Multitask Language Understanding benchmark had 540 billion parameters. By 2024, Microsoft’s Phi-3-mini reached the same threshold with 3.8 billion parameters, a 142-fold reduction in model size. (Stanford’s 2025 AI Index)

A parameter is one of the internal values a model uses to identify patterns. More parameters do not automatically make a tool appropriate for students, but smaller capable models can lower some technical barriers to building and distributing AI products. Deployment still involves costs, procurement, privacy, security, and school or district policy. The narrower point is this: more developers can potentially offer AI-powered products, so a polished interface is not proof that the underlying system has been adequately tested.

The technology is also developing quickly. Training compute, the amount of computing power used to train models, doubles every five months, while power use doubles annually, according to Stanford’s report. Those figures do not by themselves prove that regulation has failed to keep pace. They do support a reasonable concern that evaluation and procurement processes deserve deliberate attention rather than an assumption that oversight will catch up on its own. (Stanford’s 2025 AI Index)

Lawmakers are responding, but the response remains uneven. Stanford reported one state-level AI-related law in 2016, 49 by 2023, and 131 in the following year tracked by the report. That increase shows growing policy attention. It also means schools and families may face a changing mix of requirements rather than one clear national standard. (Stanford’s 2025 AI Index)

The risks of rapid AI development for students and families

A classroom tool can produce an incorrect explanation. A writing-feedback system can misread a student’s work. A recommendation system can reflect patterns that disadvantage particular groups. Those possibilities are not reasons to assume every AI product is harmful, but they are reasons to ask what was tested, by whom, and under what conditions.

The available evidence shows that safeguards do not remove every problem. GPT-4 and Claude 3 Sonnet were designed with measures to curb explicit bias, yet Stanford reported that advanced language models continued to show implicit bias. The models more often associated women with humanities rather than STEM fields and favored men for leadership roles. (Stanford’s 2025 AI Index)

That matters when a tool is used for more than brainstorming. A student may be able to check an AI-generated list of essay topics with a teacher. It is much more serious if an automated system helps shape a student recommendation, placement decision, discipline process, or evaluation without meaningful human review.

The broader record of reported incidents adds another warning. The AI Incidents Database recorded 233 AI-related incidents in 2024, a record high and a 56.4% increase over 2023. The incidents included deepfake intimate images and chatbots allegedly implicated in a teenager’s suicide. (Stanford’s 2025 AI Index)

These examples do not establish that a particular tutoring app or classroom assistant will cause the same harm. They do establish that the risks of rapid AI development include harms involving minors, not only abstract technical failures. Organizations surveyed by McKinsey also identified inaccuracy, regulatory compliance, and cybersecurity as concerns, with 64%, 63%, and 60% of respondents citing them respectively. (Stanford’s 2025 AI Index)

For educators and families, “the company says it is safe” should be the beginning of a review, not the end of one.

AI safety regulation versus innovation: what policy actually provides

Current policy includes guidance, research, and proposed standards, but those categories should not be confused with mandatory requirements.

Early last year, the U.S. AI Safety Institute at the National Institute of Standards and Technology released the second public draft of NIST AI 800-1, guidance for managing misuse risks in dual-use foundation models. A foundation model is a broad AI model that can support many different applications. The draft described voluntary practices for identifying, measuring, and reducing risks across the AI lifecycle. NIST said it incorporated feedback from more than 70 industry, academic, and civil society experts and added material on cyber and chemical or biological risks. (NIST)

Voluntary guidance can help developers and institutions ask better questions. It is not the same as a rule requiring every education vendor to meet a defined threshold before selling a product to a school. That distinction matters when a vendor describes its process as “compliant” without explaining which standard it followed or whether anyone outside the company evaluated the result.

The federal picture also shows why schools cannot assume that a single agency has settled the issue. Stanford reported that U.S. federal agencies introduced 59 AI-related regulations in 2024, while U.S. private AI investment reached $109.1 billion. Those figures describe policy activity and investment, not a guarantee that any classroom product has been tested for student use. (Stanford’s 2025 AI Index)

Regulation can impose costs, slow experimentation, and make it harder for small education providers to compete. That objection deserves a serious response. A rule designed for a system that makes high-stakes recommendations should not automatically impose the same burden on a low-risk tool that helps a teacher organize a lesson.

Still, the answer cannot be that speed is always safer. Schools need risk-based AI governance and innovation, with stronger review when a tool evaluates student work, recommends interventions, or processes sensitive information.

What responsible AI standards should require in schools

At minimum, responsible AI standards for education should address:

• Clear disclosure: Students and families should know when they are interacting with an AI system and when AI has helped produce feedback or recommendations.
• Data limits: Vendors should explain what student information is collected, how long it is retained, and whether it is used to train models.
• Independent testing: Schools should be able to review evidence about accuracy, bias, security, and age-appropriateness. A benchmark score is evidence to consider, not a safety guarantee.
• Human review: AI should not make a final decision about grades, placement, discipline, or student recommendations without qualified human oversight.
• Correction procedures: Teachers, students, and families should have a clear way to report an error and request review.
• Honest documentation: Vendors should distinguish independent testing, internal testing, and marketing claims.

Evaluation tools are improving, but they remain limited. Stanford reported that standardized responsible-AI benchmarks for large language models were previously uncommon, while newer tools such as HELM Safety and AIR-Bench are beginning to fill that gap. The Foundation Model Transparency Index rose from 37% in October 2023 to 58% in May 2024, an improvement that still leaves substantial room for more disclosure. (Stanford’s 2025 AI Index)

A benchmark can test a particular behavior under particular conditions. It cannot prove that an app handles a school’s data appropriately, works equally well for every age group, or deserves authority over a student decision. Schools should use evaluations as one part of procurement, alongside teacher judgment, privacy review, accessibility checks, and a written process for correcting mistakes.

The strongest argument against stronger safeguards is competitiveness. U.S.-based institutions produced 40 notable AI models in 2024, compared with 15 from China, while the performance gap between the top and 10th-ranked models narrowed from 11.9% to 5.4% in one year. (Stanford’s 2025 AI Index) The narrowing gap does not prove that smaller or more carefully governed systems will always compete successfully. It does weaken the claim that only a race toward larger systems can produce meaningful progress.

Schools and families do not need to choose between useful experimentation and responsible standards. They need to match the level of review to the level of risk.

Before adopting an AI tool, ask:

1. What student data does it collect and retain?
2. Is student data used to train the model?
3. Who tested the tool for accuracy, bias, security, and age-appropriateness?
4. What happens when it produces a false or biased result?
5. Can a teacher override it?
6. Does the school or district have a written policy covering its use?

If those answers are unclear, pause before putting the tool in front of students. That pause is not opposition to innovation. It is a basic condition for using AI in education with care.