Research

← Back to Guides

Good research is not only about having clever ideas. It is about asking meaningful questions, finding reliable ways to answer them, understanding the evidence deeply, and communicating the results clearly.

A good researcher should gradually develop three abilities:

1. To identify important and answerable questions.
2. To use appropriate methods to investigate those questions.
3. To explain the findings in a way that others can understand, evaluate, and build upon.

How to Pick a Good Research Question

The quality of a research question can be evaluated as a function of three main factors: novelty, feasibility, and significance.

These correspond to three basic questions:

1. Novelty: Do we already know the answer?
2. Feasibility: Can we realistically find the answer?
3. Significance: Does the answer matter?

A good research question usually sits at the intersection of these three dimensions. If a question is novel and significant but not feasible, it may be too ambitious. If it is feasible and novel but not significant, it may not be worth pursuing. If it is significant and feasible but not novel, it may be useful engineering or replication work, but not necessarily strong research.

Novelty: Do We Already Know the Answer?

A research question should not simply repeat what is already known, unless the goal is replication, verification, or application to a genuinely new setting.

To evaluate novelty, ask:

1. Has this question been directly answered before?
2. Has a very similar question been studied in another field or context?
3. Are we proposing a new problem, method, explanation, dataset, or evaluation setting?
4. If the work is incremental, is the increment meaningful enough?

Novelty does not always mean inventing something completely new. It can come from a new perspective, a better formulation, a cleaner explanation, a stronger empirical result, or applying an existing idea to an important unsolved problem.

Feasibility: Can We Find the Answer?

A research question must be answerable with the resources, time, data, methods, and skills available.

To evaluate feasibility, ask:

1. Do we have access to the necessary data, tools, models, participants, or computational resources?
2. Can we design experiments or analyses that would provide convincing evidence?
3. Is the scope appropriate for the available time?
4. Do we have, or can we learn, the technical skills required?
5. What are the biggest risks, and can we reduce them early?

A good research question should be ambitious enough to be interesting, but concrete enough that progress can be made.

Significance: Does It Matter?

A research question should matter to someone: the research community, practitioners, users, policymakers, or society more broadly.

To evaluate significance, ask:

1. If we answer this question, who would care?
2. What would become possible if we knew the answer?
3. Does it challenge an assumption, improve a method, explain a phenomenon, or solve a real problem?
4. Would the answer change how people think, build systems, make decisions, or conduct future research?
5. Is the problem central or peripheral to the field?

Significance can be theoretical, practical, methodological, or societal. The strongest research often has more than one kind of significance.

A Fourth Question: What Is the Contribution?

In addition to novelty, feasibility, and significance, it is useful to ask: What exactly will this research contribute?

Possible contributions include:

1. A new theory, concept, or explanation.
2. A new method, model, algorithm, or system.
3. A new dataset, benchmark, or evaluation protocol.
4. A careful empirical finding.
5. A replication or correction of previous work.
6. A synthesis that organizes existing knowledge in a clearer way.
7. A practical tool or design implication.

A vague topic is not yet a research question. “Large language models in education” is a topic. “How does providing worked examples generated by large language models affect novice programmers’ debugging performance?” is closer to a research question.

Stages of Research

Research can be understood as a cycle with four major stages:

1. Information collection.
2. Information understanding.
3. Information utilization.
4. Information presentation.

These stages are not strictly linear. Good research often requires moving back and forth between them.

1. Information Collection

Information collection means gathering the materials needed to understand the problem and investigate the question.

This includes:

1. Reading papers, books, technical reports, and documentation.
2. Identifying datasets, benchmarks, tools, and prior methods.
3. Collecting empirical data through experiments, surveys, interviews, logs, simulations, or observations.
4. Tracking related work, open-source implementations, and community discussions.

The goal is not to collect everything. The goal is to collect enough relevant, reliable, and high-quality information to make informed decisions.

Good Practices

1. Start broad, then narrow down.
2. Keep organized notes and references.
3. Record where information came from.
4. Distinguish between primary sources, secondary sources, and informal opinions.
5. Pay attention to both what is known and what remains uncertain.

Common Mistakes

1. Reading endlessly without forming a question.
2. Only reading papers that support your initial idea.
3. Ignoring older or adjacent-field work.
4. Trusting claims without checking evidence.
5. Losing track of sources.

2. Information Understanding

Information understanding means making sense of what you have collected.

This includes:

1. Understanding the key concepts, assumptions, methods, and results.
2. Comparing different papers or approaches.
3. Identifying agreements, contradictions, and gaps.
4. Understanding why a method works, not just how to run it.
5. Separating evidence from speculation.

A useful test of understanding is whether you can explain the idea clearly to someone else without relying on the original wording.

Good Practices

1. Summarize each important paper in your own words.
2. Ask what problem the paper solves and what it does not solve.
3. Identify the assumptions behind each method.
4. Reproduce small examples when possible.
5. Draw diagrams, tables, or comparisons to clarify relationships.

Common Mistakes

1. Confusing familiarity with understanding.
2. Copying terminology without knowing what it means.
3. Accepting conclusions without examining the evidence.
4. Focusing only on details while missing the main idea.
5. Treating all papers as equally reliable.

3. Information Utilization

Information utilization means using what you have understood to produce new insight, evidence, or artifacts. This is where research becomes active rather than passive.

This includes:

1. Formulating hypotheses or research questions.
2. Designing studies, experiments, models, or systems.
3. Implementing methods and running analyses.
4. Comparing alternatives.
5. Interpreting results.
6. Revising the question or method based on evidence.

Good Practices

1. Define success criteria before running experiments.
2. Start with simple baselines.
3. Keep experiments reproducible.
4. Change one important factor at a time when possible.
5. Record failed attempts and negative results.
6. Use evidence to update your beliefs.

Common Mistakes

1. Running experiments without a clear question.
2. Cherry-picking results.
3. Ignoring baselines.
4. Overfitting to a benchmark.
5. Treating implementation success as research success.
6. Drawing conclusions that are stronger than the evidence supports.

4. Information Presentation

Information presentation means communicating the research clearly and honestly.

Research is not complete until others can understand what you did, why it matters, how you did it, and what can be concluded from it.

This includes:

1. Writing papers, reports, theses, blog posts, or documentation.
2. Creating figures, tables, diagrams, and slides.
3. Presenting talks or posters.
4. Explaining the contribution, method, results, and limitations.
5. Making code, data, or materials available when appropriate.

Good presentation is not decoration. It is part of the thinking process. If you cannot explain the work clearly, you may not fully understand it yet.

Good Practices

1. State the research question clearly.
2. Explain the motivation before the method.
3. Make the contribution explicit.
4. Present evidence honestly.
5. Discuss limitations and threats to validity.
6. Use figures and examples to help readers understand.
7. Write for the reader, not for yourself.

Common Mistakes

1. Hiding the main idea under unnecessary complexity.
2. Overclaiming results.
3. Presenting only positive findings.
4. Assuming readers know the context.
5. Using vague phrases such as “better,” “effective,” or “significant” without explanation.
6. Treating writing as the final step instead of part of the research process.

A Simple Research Checklist

Before committing to a project, ask:

1. What is the research question?
2. Why does it matter?
3. What is already known?
4. What is unknown or unsatisfactory?
5. What is our proposed contribution?
6. What evidence would convince us?
7. What data, tools, or methods do we need?
8. What are the main risks?
9. What would be a minimal publishable or useful result?
10. How will we communicate the finding clearly?

Final Advice

Good research requires curiosity, discipline, and honesty.

Curiosity helps you ask interesting questions. Discipline helps you make steady progress. Honesty helps you respect evidence, acknowledge uncertainty, and avoid fooling yourself.

A good researcher is not someone who is always right. A good researcher is someone who can ask better questions, learn from evidence, revise their beliefs, and explain their thinking clearly.

← Back to Guides