Editorial methodology
How We Test & Review Coffee Equipment
Dr. Coffee publishes two kinds of review: pieces based on specifications, documentation and published sources, and, once we have used a product ourselves, pieces based on hands-on testing. Every review carries a label that says which one it is, before you ever see a score.
Short answer: we score four weighted criteria per product category, label every review by its evidence level (Research Reviewed, Hands-On Tested, or Not Yet Rated), and never let affiliate relationships touch a score. See a worked example in our Breville Dual Boiler (BES920) review.
Evaluation process
The stages we actually follow for a review, in order. Stages marked Proposed describe a procedure awaiting editorial approval, not a practice in place today.
-
Product selection Confirmed
A product enters our catalog because it fits an existing category in our taxonomy (such as semi-automatic espresso machines) and there is enough public documentation to describe it accurately.
-
Specification and evidence research Confirmed
We gather manufacturer documentation, published third-party tests and owner-reported patterns, and record what we could not verify as missing data rather than guessing a value.
-
Hands-on evaluation, when performed Proposed
When our team has used a product directly, findings are dated and linked to a test note. This is not yet established practice for every review; see Evidence Levels below for what “Hands-On Tested” requires.
-
Scoring and comparison Confirmed
Each criterion is scored against our category rubric and combined by its weight into one editorial score, shown alongside the weight breakdown so the number is never a black box.
-
Editorial review Proposed
A second editorial pass checking scores, claims and evidence labels before publication. The internal structure of this review has not yet been formalized in our approved sources.
-
Updates and corrections Confirmed
Reviews are revisited when prices, specifications or availability change; see Updates & Corrections below.
Review evidence levels
Every review carries exactly one of these three labels. We never imply a product was physically tested unless it carries the Hands-On Tested label.
Our team used it
Requires that someone on our team physically used the product, with a dated test note linked from the review. This is the highest evidence level and the rarest label today.
Documented, not tested by us
Based on manufacturer specifications, published third-party tests and owner feedback. We did not test the product ourselves, and the review says so directly.
Insufficient evidence
A review is planned, but there is not enough documentation or testing yet to responsibly assign a score. The product page withholds a number rather than estimate one.
Editorial scoring system
This is the live rubric for espresso machines, shown with the Breville Dual Boiler (BES920) as a worked example. Each product family has its own criteria and weights.
The overall score is reproduced here only to demonstrate the formula below. This page does not change any product’s score.
Shot quality and consistency, steam power and recovery, and how much of the result depends on the machine rather than the grinder. Basis: manufacturer documentation and published third-party tests.
Performance against current listed prices in the shopper’s region, what else must be bought, and substitutes at other price points. Basis: current listed offers and catalog substitutes.
Materials, boiler and pump design, serviceability, warranty terms and evidence of long-term failure patterns. Basis: specifications and owner-reported patterns.
Learning curve for a first-time home barista, interface clarity, and cleaning or descaling effort. Basis: manual, interface documentation and published reviews.
| Criterion | Weight | Score |
|---|---|---|
| Brewing Performance | 35% | 9.0 |
| Value for Money | 25% | 7.8 |
| Build Quality & Reliability | 20% | 8.8 |
| Ease of Use | 20% | 8.2 |
| Overall editorial score | 100% | 8.5 |
Proposed Provisional weights
Weights above are provisional and apply to the espresso machine category only; each product family (grinders, drip brewers, scales and accessories) supplies its own criteria and weights without changing this formula.
How missing information is handled
If a criterion cannot be scored responsibly (no reliable documentation and no hands-on use), that criterion is marked Not Rated and the overall score is withheld rather than calculated from three out of four criteria. We do not fill a gap with an estimate.
Editorial score vs. visitor rating
The editorial score above is written by the Dr. Coffee team and is never influenced by visitors or merchants. A separate visitor rating (stars, from reader reviews) appears on product pages but is never averaged into the editorial score, and the two are always labeled separately.
Category-specific evaluation
Criteria differ by equipment type because the questions a buyer actually asks differ. Espresso machines and drip brewers are not judged on the same rubric.
| Category | What differs | Status |
|---|---|---|
| Espresso machines | Brewing performance, value, build & reliability, and ease of use, weighted 35/25/20/20. | Confirmed, provisional weights |
| Coffee grinders | Grind quality and burr/build consistency replace brewing performance as the dominant factor. | Placeholder, not yet reviewed |
| Drip coffee makers & brewers | Brew temperature consistency, batch capacity and carafe retention are expected to matter more than portafilter-era criteria. | Proposed, pending approval |
| Scales & accessories | Precision, response time and durability are expected to replace brewing-specific criteria entirely. | Proposed, pending approval |
Only the espresso machine rubric above is live. Specific testing measurements for other categories (grind particle distribution, brew temperature stability, and similar) are not established practice yet and should be treated as proposed protocols pending editorial approval.
Evidence, limitations & uncertainty
Manufacturer specs vs. independent evidence. A manufacturer’s published number (a pump rating, for example) describes the component, not necessarily the result in the cup. We note this distinction directly in the review rather than treating a spec as a finding.
When evidence conflicts. If manufacturer documentation and independent sources disagree, we say so in the review rather than silently picking one.
Limitations of research-only reviews. A Research Reviewed score reflects documented capability, not our own extraction tests, timed usability, or measured warm-up. Where we have not measured something, the specification shows it as missing data rather than an invented figure.
Why missing data stays missing. A blank field with a “Missing data” note is more honest than a plausible-looking number we did not verify. We do not generate fictional precision to fill a table.
When products stay Not Rated. A product remains Not Yet Rated until there is enough documentation, or hands-on use, to support a defensible score across all four criteria.
Buying guide selection
Review evidence feeds our Best Of guides, but a guide pick is an editorial judgment, not an automatic ranking by score.
- Overall product score — the weighted editorial score described above, specific to one product.
- Suitability for a particular user — whether that product’s strengths match a specific buyer’s needs (counter space, budget, milk-drink habits).
- Editorial buying-guide selection — the pick an editor makes for a named use case (such as “best for small kitchens”), informed by score but not dictated by it.
The highest-scoring product in a category is not automatically the best choice for every buyer; a lower-scoring, less expensive, or more compact option can be the right Best Of pick for a specific need.
Editorial independence
Affiliate compensation and editorial evaluation are kept separate: a retailer relationship, or the presence of an affiliate link, never changes a score or a recommendation. Scores are set before offers are attached to a page, and nothing about an offer’s commission rate factors into the rubric above.
Updates & corrections
Review dates, product revisions and availability changes are expected to trigger a revisit of the affected page. Proposed A formal cadence for re-checking reviews (for example, a fixed interval after publication) is not yet an established policy and is pending editorial approval.
Factual corrections are made directly to the review when an error is identified; our approved sources do not yet establish a public changelog or correction-notice format, so that remains a production requirement rather than a current practice.
Frequently asked questions
What does the editorial score mean?
A weighted average of four criteria specific to the product’s category, written by the Dr. Coffee team and never influenced by visitors or merchants.
What’s the difference between Hands-On Tested and Research Reviewed?
Hands-On Tested means our team physically used the product, with a dated test note. Research Reviewed means the score is based on documentation and published sources, with no claim of physical testing.
Do affiliate relationships affect scores?
No. Scores are set independently of any retailer relationship or affiliate commission.
Can a score change over time?
Yes, if new evidence, a product revision, or corrected information changes what a criterion should reflect.
What happens to a product with no evidence yet?
It is labeled Not Yet Rated and shown without a score until there is enough documentation or hands-on use to support one.