Case Study: Evaluation of AI-Generated Transcripts vs. Human Transcription

Project Overview

A recent project involved a client requesting us to fix transcripts of 1:1 interviews from a leading AI transcription provider instead of transcribing them from scratch using the original audio files. The objective was to determine the effectiveness and cost-efficiency of editing AI-generated transcripts compared to professional human transcription.

Project Details

We read through 15 AI-generated transcripts whilst listening to the original audio files and made any amendments where necessary. Our transcribers were asked to change any US spellings to UK spellings, remove stutters, inaccuracies and replace any words the AI gets wrong. They also had to amend any incorrect punctuation or voice identification if the AI had included these. We did not make any changes to the AI-generated timestamps to help save time and money for the client.

AI-generated transcripts often require substantial manual corrections before they can be used.

Findings

Out of the fifteen audio files, only two files benefited from AI-generated transcripts being checked rather than being transcribed from scratch by a professional human transcriptionist.

We found that the AI had completely missed out some sections of speech on a couple of the transcripts which meant the AI-generated timestamps did not match the audio so were of no use to the client. Speaker identification had also been missed on some of these files so the formatting was not consistent across the project and where it was included, it would at times get the change of speaker incorrect which added to the correcting time and cost.

This aligns with our broader experience that AI-generated transcripts often require substantial manual corrections before they can be used. The only exception to this may be high-quality dictation files where AI tends to be more accurate than usual.

Key Insights

• Accuracy: AI transcripts often contain critical errors, including misinterpretations and omissions, which can compromise the integrity of your content and require significant manual corrections.
• Time Efficiency: Correcting AI transcripts frequently exceeds the time it takes to transcribe from scratch, leading to delays and increased workloads.
• Cost Analysis: Only two files were found to be more efficient to correct than to transcribe from scratch. The cost difference was approximately £500 + VAT across all 15 files which is quite significant. When it comes to accurate transcription of more than one voice, human professional transcription is still far more cost-effective.

Conclusion

Our findings indicate that while AI transcription technology is advancing, it still falls short in producing reliable transcripts for most audio files. Professional human transcription remains more accurate and cost-effective for most cases, except for clear dictation files where AI can perform adequately. This is especially true for group discussions, which AI cannot handle effectively yet, making professional transcription indispensable for accurate multi-speaker scenarios.

This case study highlights the importance of choosing the right transcription method based on the quality and type of audio. We continue to recommend professional human transcription for most projects to ensure accuracy and efficiency. Clients who need a predictable and financially stable way of working due to budget restraints etc. should opt to transcribe any audio from scratch as this is based on a cost per minute of audio rate so you will know exactly what the cost will be before commencement.

If you have never used a transcription agency before, or are looking to change your existing provider, please do contact us on 0800 158 3733 or office@mcgowantranscriptions.co.uk.

Data correct as of 12th July 2024.

Professional human transcription remains more accurate and cost-effective.

---

McGowan Transcriptions is an award-winning company that has been in the transcription industry for over 30 years. We provide the highest quality transcripts in the business and a second-to-none service, offering our clients clearly formatted, accurate transcripts with turnaround times to within 4 hours.

We specialise in group discussions and face-to-face interviews, covering B2B, B2C, technical, medical and legal material.

Data security is of paramount importance to us so we never outsource our projects, have UK servers, Webroot encryption and we are GDPR and Cyber Essentials Certified.