What can happen when employers use AI to write job posts

open share links
close share links

An employer posts a new job, summarizing the key responsibilities and required qualifications. It describes the workplace culture and employee benefits. People apply, sending in their resume, a cover letter, and perhaps letters of reference.

In the end, many words move back and forth.

“Given the amount of written material involved in this matching process between jobs and the people applying for them, it’s always seemed like generative AI would be well suited to help out,” said Emma Wiles, SM ’22, PhD ’24, a digital fellow at the MIT Initiative on the Digital Economy. “But that’s not really what we find.”

With MIT Sloan associate professor John Horton, Wiles conducted an online experiment in which some employers were given access to an AI assistant to help them draft job posts. In a working paper detailing the results, they share that the AI tool does make the job-posting process more efficient, but those gains are overshadowed by costs to job seekers in the form of low-quality job postings and wasted time. 

More job posts, but not more jobs 

The experiment involved a website designed to help businesses fill remote work positions — jobs like computer programming, copy editing, or graphic design, ranging from short gigs for a few hundred dollars to longer term, full-time assignments.

During the period of the experiment, employers that joined the website to post jobs for the first time were randomly given the option of using AI to help draft their post. In those cases, the companies would summarize what they were looking for in two sentences, and AI (the GPT-3.5 Turbo large language model) would write a first draft of the full job post, which employers could then edit.

1
5
%

In an online study, job posts written with AI were 15% less likely to lead to a hire.

Roughly 75% of employers chose to use AI if it was offered, and most of them accepted the draft copy without making edits to it. This process cut the time required to write posts by 44%, on average. From the perspective of the employers, the AI proved quite useful, and it made them 19% more likely to post a job. 

Although the number of postings increased, the researchers did not find an increase in actual job matches. In fact, job posts written with AI were 15% less likely to lead to a hire. 

“When we made it this easy to write job descriptions, we found that it encouraged employers with no real interest in hiring to post jobs — employers that simply wanted a way to get a sense of the applicant pool, to see who’s out there,” Wiles said. 

The researchers also found that the job posts themselves were less informative when drafted by AI. Rather than creating a post asking for a programmer with four or more years of experience who knows Python and Julia, the AI would generate something more generic, such as a post looking for an experienced go-getter with a background in software engineering.

The result of these two effects was a less efficient hiring market. Though more jobs were posted, the number of positions being filled did not increase — in part because some employers had no intention of hiring, and in part because the more generic posts led to poorer matches between the people applying and the job.

“While the technology saved employers a lot of time, it ended up wasting job seekers’ time,” Wiles said. “Employers may have an incentive to use it, but, from a central planner’s perspective, this is a costly tool to introduce to the marketplace.”

Reining in the use of AI

The researchers also found a relationship between the length of time employers spent drafting a job post and the number of people who applied for the job. In the control group, which did not have access to the AI tool, spending more time on a job post typically resulted in a longer description and more applicants.

“In the pre-AI world, there is a very strong relationship between the amount of time an employer spends on a post and its length, and so when applicants see a long, detailed job post, they assume that the employer is serious about the job,” Wiles said. “AI breaks this relationship.”

The AI-drafted posts, in comparison, were all roughly the same length, and those posts on which employers spent the least amount of time actually attracted more applicants. A once-useful signal of employer intent disappears from the labor market when AI enters the mix.

This experiment, Wiles noted, took place in mid-2023, when generative AI tools were relatively new. While there’s no definitive information on the state of adoption, anecdotal evidence suggests that these tools are now pervasive in the labor market, raising concerns about increasingly poor matching between employer and applicant.

“One potential way of getting around this is that employers could have to pay 10 cents, or a dollar — some amount of money — to use these AI tools when they create a job post,” Wiles said. Where the time it took to do this work used to deter excessive posting, now a small surcharge might do the trick. 

“This would hopefully reintroduce enough friction to turn away those that are posting jobs with no real interest in hiring,” Wiles said. “It could stem the flood of ghost posts.”

Read the paper: “Generative AI and labor market matching efficiency.”