Instructors and administrators are quite interested in a quick and painless solution to the prevalent problem of plagiarism. Software that supports identifying text similarity that could be plagiarism suggests that a single number can be derived that represents such a similarity level. Institutions sometimes rely on this number to determine sanctions. But how good are the systems at identifying problematic text? A research group with the European Network for Network Integrity recently published a multi-language test of 15 popular systems. The ramifications of the results on the use of these systems in educational settings will be discussed.