Bots and Unwanted AI Submissions

Bots are a reality of the internet. Every website is affected by them on a daily basis. AI submissions are a relatively new phenomenon that's quickly affecting the entire world of publishing. This article aims to tell you how Duosuma (Duotrope's submission manager platform) is handling these issues and to provide advice on how to mitigate bots and AI from affecting your submissions (whether you use Duosuma or not).

Defining Terms: Bot and AI

A bot is an automated script or process that navigates the internet and interacts with websites (ex: submitting a contact form or scraping the website's text).
For the purpose of this article, AI refers to LLMs (large language models) and other machine learning services that can generate text or images based on a prompt.

What is Duotrope doing?

Duotrope has always striven to have robust security. We have several layers in place, from the basics (like SSL), to using a state-of-the-art security service (Cloudflare) as middleware, to developing our own industry-specific solutions in-house. That said, no solution is fool- or bot-proof, and all solutions that attempt to identify bots-vs-humans have false-positives. We try to provide a balance between security and a good human experience.

How many submissions are made by bots or using unwanted AI?

Bots: We have investigated several suspicious submissions on our submission manager platform, Duosuma, and we were not able to determine definitively that they were submitted by a bot rather than a human in any of those cases. All evidence suggested that humans made the submissions.

AI: People can submit AI-created material (and they may even lie about it), and people are also capable of writing material that sounds like it was fabricated by an AI. The hard truth is that it's nearly impossible to tell AI-created creative work apart from amateur writing. That issue can be compounded further when the submitter is not an expert speaker of the language being submitted, especially if they use a translation tool.
More than half of the cases we've investigated have been submissions by people in foreign countries who are not native speakers of English. It's possible that their English language skills simply aren't up to the standards the publication is accustomed to, or that they've run their work through a translation tool that flattened voice, artistry, and nuance.

How does Duosuma mitigate bots?

By the time someone has created an account and started a submission on Duosuma, several security checks and human-vs-bot checks have already been passed. In addition to that, some submitters will have to answer a short, simple Q&A (CAPTCHA) to show that they are human. These challenges will be given to more suspicious requests more frequently, but even trusted users will be challenged occasionally.
We use a custom Q&A rather than an image-based CAPTCHA for several reasons. We find they are less of a hassle for humans to solve while being more challenging for bots to solve, and they meet accessibility standards.

What tools does Duosuma offer to mitigate unwanted AI submissions?

Also, Duosuma offers all our submission manager accounts the following options:

• Setting whether AI-Assisted Works are permitted
  This can be set in Account settings. If prohibited, submitters will have to affirm that they did not use AI/machine learning tools in the creation of the work. If they don't affirm this, they won't be able to submit.
• Request a link to their Duotrope Profile
  Duotrope profiles provide, at a minumum, the submitter's name, tagline, and bio, as well as a tag that states how long they've been using Duotrope/Duosuma. Duotrope subscribers can also add photos and their publication histories. (Note: When you request a Profile link on an anonymous submission, only people on your team with "Can Reveal identity" permission will have the option of revealing the identifying information, including the Profile link.)
• Custom Questions
  You can set up Custom Questions that require specific answers. If the answer provided by the submitter isn't in the allowed list, they won't be able to proceed with their submission.
• Implementing a Ban List (for submitters breaking your rules)
  See the guide to banning submitters.
• Reporting Violations
  Team members at Editor level or above can report any submission as a possible violation to Duotrope. This gives us permission to fully investigate the submission and take appropriate action, which may include banning the submitter completely from using Duotrope/Duosuma. Use of a bot to submit content is against our Terms of Use.

What are the best ways of spotting AI/LLM submissions?

Let's look at a few approaches and their validity. (Spoiler alert: There is currently no reliable way of detecting AI use. There are some up-and-coming tools that might help demonstrate that you didn't use AI. We're excited about the prospect of those becoming mainstream, and we will update this article as new approaches become available.)

• The submitter's Bio and/or Cover Letter seem "off" in some way: stilted, robotic, or not having the appropriate tone for the work being submitted.
• AI-created material tends to have the following characteristics:
  • lacks personality
  • is very repetitive
  • has awkward flow
  • doesn't demonstrate real life experience
  • has superficial or shallow emotional depth
  • has plot gaps or holes
  • lacks voice or has inconsistent voice
• If you look up the submitter, what is their online footprint? It's not hard for a bot to create fake social media profiles, too, but they won't be well used. On the other hand, keep in mind that some real people, including the author of this article, don't have much of a social media presence.
• You can run the submitted text through an AI detection service (which use AI, ironically), but be aware that false-positives are extremely common with all AI detection tools. They use probability, not analysis. Also, those tools are best suited to detecting AI-generated nonfiction, such as academic essays, and are even less reliable when it comes to fiction and poetry. These detection tools mostly muddy the waters, especially for creative writing. Even for nonfiction, the accuracy is low and even worse for non-native English speakers. They've flagged famous works, such as the United States Constitution, as AI-written. In short, AI detection services are not proof, one way or the other.
• If you ever really like a piece and want to publish it, but you're concerned about AI use, err on the side of contacting the submitter and asking if they can provide a file that shows their revision history or early drafts. Google Docs and Grammarly save revision history and so do many others. A human's revision history will be sporadic, whereas AI's happen in big chunks. Messy first drafts can also demonstrate (but not prove) human authorship.
• You may have heard that certain punctuation idiosyncrasies — such as the em-dashes used here — are definitive signs that someone used an AI. That simply isn't accurate. Humans are perfectly capable of typing em-dashes, too, and many writers (including the human author of this article) are quite fond of them. Punctuation is part of an individual's writing style. If you ever hear "_____ is a sure sign of AI," take it with a very large grain of salt. If it were that simple, this article and detection tools wouldn't be necessary.

How can we set up our submission calls to trick bots?

There are a few ways of doing this, but none is absolutely bot-proof. Many of them we've already implemented as part of the submission process on Duosuma, but if you want to take it up a notch (and especially if you also want to eliminate submissions from people who don't follow instructions) here are some other ideas:

In your Cover Letter instructions, ask for the cover letter to include something special. It could be a specific word or phrase or the answer to a question. If you're reviewing a submission and the Cover Letter doesn't include this, you can decline it outright. This will, of course, affect humans who don't follow instructions as well as bots. It's up to you to determine whether that's a good or bad thing. 

We've also seen some publications request that certain words or phrases be included in the actual submission file. While this option might be effective at mitigating bots, it's a bit more work in the review process, since you have to take the time to open the file and search for the secret word or phrase. It's also quite a bit more hassle for legitimate submitters, who may be used to re-using the same file.

The impacts of AI on our world are quickly evolving, and we will continue to do our best to prevent the negative consequences of this from affecting writers, artists, and publishers using Duotrope/Duosuma. If you have any concerns, ideas, or thoughts you'd like to share with us on this topic, feel free to contact us.

---

Go to the Duosuma Home Page.
See more Documentation.
Have any questions about this? Ask us.