Qi Sun

Prompt Iteration Log

A shareable internal tool that enables non-technical colleagues document prompt development, search prompt histories, export prompt logs, and learn from previous prompt iterations

• Details

LLM-Assisted Survey Research Workflow

A reusable workflow that combines LLM-assisted analysis with researcher validation to streamline quantitative and qualitative analysis, reduce repetitive analytical work, and establish a structured approach for survey research

• Details

Cross-Stakeholder Evaluation Framework

An evaluation framework that compares student and faculty perspectives on teaching, learning, and educational experience to identify perception gaps, align priorities, and inform institutional improvement

• Details

Longitudinal Student Outcome Measurement

A longitudinal measurement framework that combines AI-assisted data integration with researcher-led multidimensional analyses to examine multiple student outcome measures

• Details

Vibe Coding · Google Apps Script · Google Sheets ·
Google Sites · HTML · CSS · JavaScript

Prompt Iteration Log

• Problem:
As LLMs become increasingly adopted in institutional research offices, non-technical colleagues need simple ways to learn from previous prompt development and reuse validated prompts. However, prompt development is often iterative, and there was no systematic way to document why prompts changed, what problems each revision addressed, or which versions consistently produced reliable results. As a result, valuable prompts were difficult to share and were frequently recreated from scratch, making it harder for non-technical colleagues to use LLMs effectively for research tasks.• What I Built:
I built a simple and shareable internal tool consisting of two integrated components:
1) Prompt Iteration Log : Records every prompt revision together with the issue it addressed, compares each version with the previous iteration, identifies the final validated prompt, and captures metadata such as the AI model and contributor.
2) Data Viewer: Provides a searchable interface for reviewing prompt histories, identifying reusable prompts, and analyzing common iteration patterns across research projects. Users can export prompt logs to Excel.The system automatically generates summaries of prompt iteration histories, highlighting the most frequent issue types, the most iterated projects, and validated prompts for future reuse. Together, these features create a searchable knowledge base that supports more consistent and efficient AI-assisted research workflows.• Impact
- Enabled non-technical researchers to learn from previous prompt iterations
- Facilitated the reuse of validated prompts across research tasks
- Reduced the need to recreate prompts from scratch• Technologies:
Vibe Coding; Google Apps Script as a backend API; Google Sheets as the data store; Google Sites for web interface; Microsoft Teams for distribution.• Challenge:
The primary challenge was designing and implementing a web application without a traditional software engineering background. AI-generated code accelerated development but still required iterative testing, debugging, and refinement to ensure that the interface, backend logic, and data storage functioned reliably as an integrated system.• Future Improvement:
- Support keyword-based search across prompts and issue categories.
- Add dashboards to visualize prompt performance, common failure types, and iteration trends over time.• Reflection:
This project demonstrated that non-technical researchers can use AI-assisted development to build practical internal research tools. More importantly, it showed that effective prompt engineering depends not only on developing effective prompts but also on systematically capturing, validating, and sharing the knowledge generated through iterative refinement.

• Return

• Next Project

Tool:
https://sites.google.com/view/oir-prompt-log/home

Source Code:
https://github.com/susanqisun/Prompt-Iteration-Log

Figure 1. Prompt Iteration Log interface

Figure 2. Data Viewer dashboard

Survey Research · Quantitative Analylsis ·
Qualitative Analysis · SPSS syntax· Claude ·
human-in-the-loop validation

LLM-Assisted Survey Research Workflow

• Problem:
Survey research often requires repeating the same analytical workflow across projects, including data cleaning, descriptive summaries, subgroup comparisons, statistical testing, and qualitative comment analysis. These analyses are time-consuming to reproduce using traditional tools such as SPSS or Python, while ad hoc use of LLMs can introduce calculation errors, inconsistent outputs, and unsupported interpretations.• Focus:
To develop a reusable survey research workflow that combines LLM-assisted analysis with researcher validation to streamline quantitative and qualitative analyses while maintaining research quality.• What I Built:
Developed a reusable LLM-assisted survey research workflow that automates repetitive analytical tasks while preserving human oversight. The workflow generated structured multi-sheet Excel reports, including descriptive summaries, subgroup comparisons, crosstab analyses, statistical results, qualitative theme summaries, and trend analyses.• Impact
- Reduced analysis time
- Improved workflow consistency
- Established a structured approach for integrating LLMs into survey research• Challenge:
The greatest challenge was ensuring research validity rather than simply generating results. LLMs could produce plausible outputs while misunderstanding variable definitions, subgroup coding, or analytical instructions. To address this, I systematically verified LLM-generated analyses against SPSS outputs before findings were incorporated into research reports.• Future Development:
Expand the workflow with standardized prompt libraries, validation checklists, and evaluation metrics for assessing LLM reliability across different survey designs. Future work will also extend qualitative coding to support substantially larger volumes of open-ended responses.• Reflection:
This project changed how I approach survey research. Instead of spending most of my time manually analyzing data, I now focus on designing the workflow, defining analytical requirements, and validating results. It showed that LLMs are most valuable when they support - not replace - research judgment. Also, I learned that successful human-AI collaboration depends less on the model itself and more on the research process around it. Clear instructions, structured workflows, and careful validation are essential for producing trustworthy results.

• Return

• Next Project