About Duotrope's Handling of Outliers
The Basics: What is an "Outlier"?
An outlying observation, or outlier, is one that appears to deviate markedly from other members of the sample in which it occurs.
How does Duotrope determine what submission reports are considered outliers?
Over the course of several years, we developed a complex algorithm to determine outliers, and we periodically make improvements to it. The algorithm looks at a variety of factors involving both the publisher/agents' statistical patterns and the members' statistical patterns.
Also, note that all submission reports are constantly being re-evaluated as statistical patterns change. A report that was not originally considered an outlier can be considered an outlier at a later date, and vice versa.
What does Duotrope do with outliers?
By default, outliers are not included in the Response Statistics on our listings. However, there is an option to include outliers in the statistics. Simply check the "Include outliers" box and click the "Display" button.
One (or more) of my submissions reports has been flagged as an outlier...
Note: Just because one of your reports was considered an outlier doesn't mean that they all are.
What can I do?
Apart from making sure your submission records are as complete and accurate as possible, there is no need to do anything. The system is working as intended.
Also, it's better to report a submission as "Never Responded" when that option becomes available (indicated by orange or purple color coding) than to hold onto pending submission reports for years. You can always update those reports later if/when you do get a response. Learn more about when to query and when to give up waiting for a response.
Will you unflag it?
Our admins cannot change the status of outliers as they are under constant evaluation by the outlier algorithm. We do not manually handle outliers.
Can you tell me why it was considered an outlier?
No, we can't. Our algorithm requires intense computing. It's not something that we can calculate by simply looking at the reports in question.
Do you think I'm lying? I swear I'm not!
No, we don't. It simply means that your report is outside of statistical norms.
I've been lucky, and my acceptance ratio is high. Now it seems all or most of my reports are considered outliers. What should I do?
Congratulate yourself! You haven't just been lucky; you've been highly successful! If Stephen King/Roxanne Gay/Kazuo Ishiguro were to use Duotrope, their submission reports would be considered outliers, too.
Technical Questions
I see data points counted as outliers that are within 3 sigma (3 standard deviations of the mean). What's that about?
The data Duotrope is dealing with doesn't always follow a normal distribution. We do, of course, take into consideration standard deviation, but if it doesn't make sense given the data set, we apply different, proprietary calculations that were developed specifically for the nuances of submission response data and the (mis)reporting patterns we see.
Also, data points can be excluded based on the reporting user's patterns (such as having an extremely high acceptance rate or other unusual patterns). Those data points might appear to be non-outliers when only taking into account the listing's data, but we're looking at a bigger picture as well.
Have a question? Ask us!