As consultants and professors in business and technology, we see every day how Artificial Intelligence (AI) is redefining the way businesses operate. In fact, according to McKinsey, over 70% of companies have adopted some form of AI in at least one business unit, and those leveraging AI at scale are seeing performance gains of up to 20% in areas ranging from operations to customer engagement.¹ These changes in business demand a corresponding shift in how we educate the next generation of leaders by integrating AI into the very core of our teaching at business schools.

Although this shift is generally considered exciting, as business educators we recognize that approaches to AI adoption in the business curriculum are not yet necessarily well-developed, and as such run the risk of demotivating faculty and students alike if AI is not well implemented. Any successful AI adoption in education requires that the use of AI is recognized by students as useful and enhancing their learning, and tools are developed in ways that align best with how faculty teach, and students can effectively process feedback generated by AI.

To achieve these successful outcomes requires an approach that – before implementing AI across the curriculum - explicitly enables testing AI methods and measuring specific learning goals and outcomes in the classroom. We explored such an approach at the D’Amore-McKim School of Business (Northeastern University), where we transformed the classroom into an AI Learning Innovation Lab – referred to as the D’Amore-McKim AI Strategic Hub (DASH). In this setting, each instructor can build powerful methods to advance student learning. By adopting A/B designs (comparing AI versus non-AI approaches), developing AI tools alongside users and measuring outcomes, faculty can create clarity and confidence in AI-enabled learning. Here we present our use of our AI innovation lab to learn the best ways to develop the right AI tool, understand student’s perceptions of AI and the impact AI reveals when using it as a feedback system that faculty can use when assessing students’ work.

Method and Approach

We describe studies that we ran in the classroom where we tested AI’s ability to provide feedback based upon a student’s essay performances and compared it with feedback delivered by human graders. The AI we used was Generative AI (GenAI) – AI systems capable of producing original text, analysis, and recommendations by learning from vast datasets. All feedback delivered by AI and human graders was reviewed by the faculty teaching the courses, and any required modifications made if needed. Specifically, during development of the GenAI grader, faculty, teaching assistants (TAs) and selected AI engineers iterated the grading rubric for each course, and test-graded both real and AI-generated student essays. Lead faculty reviewed these rubrics and grading feedback output for quality, and both TAs and AI engineers incorporated any required changes to improve grading. This was repeated until the faculty was satisfied with both the human and AI grading styles.

Half of the students randomly received AI feedback and the other half human grader feedback. Students then evaluated the feedback that was delivered (either by AI or the human) and their subsequent performance was measured by providing them with a very similar second essay task based on the same course material as the first task up to two weeks later (see Figure 1 for an overview of the experimental approach). This approach created a setting for DASH to obtain our own – and internally validated – insights on how and to which extent AI can be used by faculty to improve the learning journey of their students.

Figure 1: GenAI and Human Feedback for Student Assessments

Our approach also allowed us to examine the feedback abilities of AI – and acceptance of it by students and faculty – in alignment with business practices where AI is used to provide recommendations such as product reviews or job application evaluations. Promoting such alignment makes it possible to promote experiential learning further, as we could test in real-time (1) whether AI’s recommendations (feedback) can improve performance and decision-making, and (2) how AI is best developed for leaders to promote its use most effectively. First, by measuring whether business students implemented AI-generated feedback, we can see whether they embrace the recommendations made by AI and improve their performance while at the same time controlling for their perceptions of the recommendations given as valuable and reliable insights or not. Second, by developing our own GenAI rating system, we can test which procedure of development – driven by AI engineers only or by collaborative efforts between engineers and faculty - is best for students to accept and use AI-generated feedback.

We deployed a parallel AI and Human feedback protocol to over 200 students across five courses and four business disciplines, including analytics, information systems, marketing and management. Each course had assessments tasks based on standard course content that would typically appear as an assignment or test. So, what did these studies reveal with respect to the earlier mentioned outcomes that we need to be confident in so AI adoption in the business curriculum can succeed?

Findings

1. The “How” – the impact of AI tools is enhanced when engineers understand users

Successful AI adoption happens when resistance can be reduced. Achieving such outcome requires that AI tools should be developed in ways that align best with how faculty teach, and students process the generated feedback. In line with this requirement, we, first of all, observed that the AI engineers who were involved in developing rubrics, grading sample assessment tasks, and interacted with the faculty about the essay assignment (compared to those who only focused on developing the AI tool), were more familiar with the educational context under which the AI had to be used. As a result, the GenAI bot developed by those engineers resulted in grades and feedback that were more aligned with faculty expectations, Because of that alignment, faculty resisted less to the proposal to use AI and regarded the use of AI as more positive. For example, engineers who simulated the role of a grader and discussed their experiences with faculty, created code and prompts that were more immediately useful to faculty, who in turn could more readily prepare subsequent content. In contrast, engineers that did not directly grade student work before coding, tended to expect students answers to resemble course content directly, or be written in an archetypical style without consideration for differences across language or cultural references.

2. The “What” – Measure Outcomes, and Perceptions of Outcomes

Effectively adopting AI in the business curriculum requires that students perceive the AI tool as useful and improves their actual performance. These outcomes require testing how AI-assisted feedback is received by students. We found that students who received AI feedback tended to improve more compared to students that received human feedback (as measured by the difference in quality across the first and second essay tasks). Interestingly, however, perceptions of feedback didn’t always align with the reported performance improvement. While there was no mean difference between perceptions of bias in feedback between AI- and human- graded groups, the variance of perceptions was often larger for students that received AI-assisted feedback. This was true even for instances where AI-assisted feedback improved performance. Since even people who benefit from AI may exhibit greater positive and negative impressions of the experience, it is clearly not enough to achieve improvement, it is also important to achieve consistent buy-in. From that perspective, we did find that faculty were positive because it allowed them to provide extensive written feedback to students within just a few days.

3. The “Why” – On the necessity of using A/B test methods

From our studies we also learned how to deal with results that provide evidence that not all requirements for successful AI adoption are achieved in the classroom. In our studies, we found that perceptions and (improved) performance did not align well. This finding can cause worry because if perceptions among students about the usefulness of AI as a teaching and learning tool are not consistent, it could result in less enthusiasm for the use of AI and even lead them to underestimate the actual positive effect of deploying AI (remember that objectively speaking the use of AI enhanced performance more, even though perceptions were not very aligned). This observation highlights the need to identify the right conditions under which perceptions and performance of students as a function of using AI as a feedback tool align and as such promote the positive overall impact of AI. We can do so, as we illustrate in our studies, by employing A/B test designs. This can be as simple as providing a survey asking students if they are open to AI-assisted feedback compared to traditional human TA-assisted feedback or explaining the benefits of AI for students and faculty versus no explanation, and later tracking feedback satisfaction to see what the right conditions are that can lead to AI acceptance and effectiveness.

Recommendations

Our approach demonstrates effective adoption of AI is enhanced through an open application of a rigorous, experimental mindset where students and faculty are partners in creating shared understanding. Indeed, using an AI innovation lab approach such as DASH within the business school can help to develop frameworks and use AI tools in ways that students don’t just accept but actively incorporate AI into their learning and future workflows. With such an approach the integration of AI into business education marks a pivotal shift—from passive knowledge absorption to active, experiential mastery. To promote this kind of experiential learning, our approach has taught us that the classroom must evolve into a dynamic space where leaders learn not just about AI, but how to wield it effectively.

The key for effective AI adoption lies in systematically testing – by means of A/B designs - AI-driven teaching methods against traditional human-led approaches—measuring both specific learning outcomes and engagement. Does AI enhance case analysis? Speed up feedback? Deepen critical thinking? But, as our findings illustrate, the real insight are derived from assessing how these tools reshape learning: Do students retain more? Collaborate better? Develop sharper judgment? By using the classroom as analyzing both performance data and learner perceptions, schools can refine a hybrid model that amplifies human expertise with AI’s scalability. When faculty use these information-based methods, it is easier to share data-supported insights with students and fellow faculty and consequently build a shared understanding of AI-enabled classroom practices.

To harness AI’s full potential, faculty should explicitly an interdisciplinary AI team through recruitment TAs who have exposure to both AI and pedagogical context. In these early stages of AI adoption, it is common for engineering students to be recruited as research or teaching assistants to help develop AI tools. If this assistant has an overly technical background (very likely), they must experience the process of coursework development and student assessment. The most effective solutions will emerge when AI developers collaborate directly with business faculty—combining technical prowess with deep domain expertise. These interdisciplinary teams can design tools that are not just innovative, but relevant, ensuring AI aligns with real-world challenges and pedagogical goals, and is positioned well so students pick up more easily what needs to be learned.

Your classroom is your AI innovation lab and is a golden opportunity to prepare both students and faculty to adopting AI by assessing where they are in terms of their level of resistance, acceptance, and potential biases (e.g. not seeing AI as a solution, even though it objectively improves performance). By being open with students and measuring outcomes and perceptions, business schools create opportunities to accelerate common understanding of how AI can be effectively incorporated into the classroom. The result will be the use of AI tools that faculty trust, students adopt, and industry values. By co-creating with end-users, schools can move beyond gimmicks to tools that enhance decision-making, critical thinking, and problem-solving. The future of business education isn’t just about adopting AI—it’s about building it right.

References

1. McKinsey, The state of AI: How organizations are rewiring to capture value, March 12, 2025. Retrieved from: https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai