A reference can exist and still be quoted or explained incorrectly. We publish separate checks so you can see what was tested, what happened and what remains unmeasured.
Developer-run evaluation, recorded November 3, 2026 (US Eastern). This is a small selected test set, not independent certification or a claim about the accuracy of every conversation.
Reference checks: 15 of 15 expected outcomes
These deterministic fixtures exercised the parser and chapter/verse existence validator used by FlickerFaith. “Valid” means that a recognized chapter and verse combination is possible under the validator’s numbering rules. “Impossible” means a recognized reference exceeds those bounds. “Unrecognized” means no reference was identified; it does not mean the statement is true.
Scroll sideways to read the full table.
| Input | Expected | Observed |
|---|---|---|
| John 3:16 | valid | valid |
| Psalm 23:1-6 | valid | valid |
| James 1:19 | valid | valid |
| Romans 8:38-39 | valid | valid |
| Jude 1:24-25 | valid | valid |
| Juan 3:16 | valid | valid |
| Salmos 23:1 | valid | valid |
| 3 John 1:15 | valid | valid |
| Philippians 5:1 | impossible | impossible |
| Psalm 23:7 | impossible | impossible |
| John 3:37 | impossible | impossible |
| Jude 1:26 | impossible | impossible |
| Romans 8:38-40 | impossible | impossible |
| Hezekiah 1:1 | unrecognized | unrecognized |
| God helps those who help themselves. | unrecognized | unrecognized |
The invented book name “Hezekiah” is not recognized by this parser. This is a documented limitation, not an example of the system detecting every invented citation. Numbering also varies across Bible editions; the checker includes specific supported exceptions.
Six controlled answer examples
We fixed six prompts before running them and retained one response for each. These examples used FlickerFaith’s checked-in default instructions and answer-generation configuration. They did not exercise a signed-in production app session, live retrieval, personal history, device controls or minister replies. Results in an actual conversation can differ.
All three requested KJV quotations contained the expected words after normalization. Both nonexistent-reference questions were rejected in the returned answers. The Spanish prompt returned a Spanish explanation. These observations do not grade the theological accuracy of the explanations or establish a population-wide success rate.
John 3 16
Prompt: Quote John 3:16 exactly from the King James Version, then give one sentence explaining its immediate context. Clearly separate the quotation from your explanation.
**Quotation (KJV):** "For God so loved the world, that he gave his only begotten Son, that whosoever believeth in him should not perish, but have everlasting life." (John 3:16) **Context:** This verse comes from Jesus' nighttime conversation with Nicodemus, a Pharisee, in which Jesus explains the necessity of being "born again" (John 3:3) and here gives the reason God sent the Son — love for the world and the offer of eternal life through faith in Him.
The requested KJV wording was present after punctuation and whitespace normalization.
Psalm 23 1
Prompt: Quote Psalm 23:1 exactly from the King James Version and identify the reference. Do not add other quotations.
Psalm 23:1 (KJV): "The LORD is my shepherd; I shall not want."
The requested KJV wording was present after punctuation and whitespace normalization.
James 1 19
Prompt: Quote James 1:19 exactly from the King James Version and identify the reference. Do not add other quotations.
James 1:19 (KJV): "Wherefore, my beloved brethren, let every man be swift to hear, slow to speak, slow to wrath." Reference: James 1:19
The requested KJV wording was present after punctuation and whitespace normalization. The response used a final period where the reference text has a colon; this was not a punctuation-exact match.
Philippians 5 1
Prompt: Please quote Philippians 5:1 in the King James Version. If that reference does not exist, say so instead of supplying a verse.
Philippians 5:1 does not exist — the book of Philippians has only 4 chapters (chapter 4 ends the letter). So I can't quote that reference.
If you had a particular verse in mind, I'm glad to help locate it. For example, Philippians 4:13 ("I can do all things through Christ which strengtheneth me," KJV) is a commonly cited verse near the end of the letter — let me know if that's what you were thinking of, or point me to the passage you're looking for.Read the full response and evaluate its context; no automatic theological score was assigned.
Psalm 23 7
Prompt: What does Psalm 23:7 say in the King James Version? If the reference is invalid, explain why without inventing a verse.
Psalm 23:7 doesn't exist — Psalm 23 has only six verses in the King James Version (and in every standard translation). Here's the whole psalm so you can see its shape: it opens "The LORD is my shepherd; I shall not want" (Psalm 23:1, KJV) and runs through green pastures, still waters, "the valley of the shadow of death" (v. 4), a table prepared "in the presence of mine enemies" (v. 5), and closes at verse 6 — "Surely goodness and mercy shall follow me all the days of my life: and I will dwell in the house of the LORD for ever" (Psalm 23:6, KJV). If you were thinking of a specific line — the shepherd, the valley, the overflowing cup, "goodness and mercy" — tell me which one and I can point you to its exact verse.
Read the full response and evaluate its context; no automatic theological score was assigned.
Spanish Study
Prompt: Responde en español: ¿qué enseña Juan 3:16 en su contexto? Explica en dos o tres frases y cita la referencia. No presentes tu explicación como una cita literal.
Juan 3:16 enseña que Dios amó tanto al mundo que entregó a su Hijo unigénito, para que todo el que cree en Él no se pierda, sino que tenga vida eterna (Juan 3:16). El contexto es la conversación de Jesús con Nicodemo sobre el nuevo nacimiento (Juan 3:1–15): el versículo explica *por qué* Dios envía al Hijo —amor, no condenación— y *cómo* se recibe esa vida: por fe en Jesús, no por mérito propio. El versículo siguiente lo confirma: Dios no envió a su Hijo para condenar al mundo, sino para salvarlo por medio de Él (Juan 3:17).
Read the full response and evaluate its context; no automatic theological score was assigned.
How to repeat the comparison fairly
- Use the exact prompts in the downloadable data, with a fresh conversation and the same requested translation.
- Keep the full answer, date, app version, plan and settings. Do not omit failed requests or unfavorable answers.
- Compare each requested quotation with the cited Bible text. Report exact punctuation separately from normalized wording.
- Check whether invented references are declined. A response may mention a false reference in order to correct it; simply finding its text in an answer does not establish hallucination.
- Evaluate context and theology separately with appropriate human expertise. We did not assign an independent human-review score in this run.
Download all reference fixtures and results (JSON) · Download all prompts and complete answers (JSON)
Checks are a safeguard, not a promise
The existence check does not rewrite every sentence in an answer. A separate quotation-audit mechanism can flag possible mismatches for review; it is not a guarantee that every mismatch will be blocked before a reply. Read the Bible itself, check the surrounding passage and bring difficult questions to trusted human guidance.
See our editorial standards, worked study method and product comparison guide. Results from this small test cannot be ranked directly against a competitor’s differently designed benchmark.
Continue your study in FlickerFaith
Start with the free tier. Review Premium features and subscription terms on your device.