Prompt Engineering Best Practices for ChatGPT Apps
Prompt engineering is the foundation of high-quality ChatGPT applications. The difference between a mediocre app and an exceptional one often comes down to how well you craft your prompts. Poor prompt engineering leads to inconsistent responses, hallucinations, and frustrated users. Excellent prompt engineering produces reliable, accurate, and contextually appropriate outputs that delight users.
In this guide, you'll learn production-ready prompt engineering techniques used by leading AI companies. We'll cover system prompts, few-shot learning, chain-of-thought reasoning, prompt templates, and evaluation strategies—all with executable Python code you can deploy today.
Whether you're building a customer service bot, a content generation tool, or a specialized domain assistant, these techniques will dramatically improve your app's performance. The examples are production-tested, scalable, and designed for real-world ChatGPT applications.
Why Prompt Engineering Matters for Production Apps
Every ChatGPT interaction begins with a prompt. The quality of that prompt determines whether the model provides a helpful response or goes off-track. In production environments, poorly engineered prompts create compounding problems:
Consistency issues: Without clear instructions, the same prompt can produce wildly different outputs across sessions, making your app unpredictable.
Quality degradation: Vague prompts lead to generic, unhelpful responses that fail to solve user problems.
Wasted tokens: Inefficient prompts consume more tokens than necessary, increasing costs and latency.
Safety vulnerabilities: Inadequate constraints allow users to bypass intended behaviors through prompt injection attacks.
Professional prompt engineering solves these problems by creating structured, repeatable patterns that guide the model toward desired behaviors. Let's explore the core techniques.
System Prompts: Defining Behavior and Constraints
System prompts establish the foundation for every conversation. They define the model's role, behavior constraints, output format, and domain expertise. A well-crafted system prompt is like a detailed job description that the model follows throughout the interaction.
Here's a production-ready system prompt builder with role templates, constraint enforcement, and format specifications:
from typing import Dict, List, Optional
from dataclasses import dataclass
import json
@dataclass
class SystemPromptConfig:
"""Configuration for system prompt generation."""
role: str
domain: str
constraints: List[str]
output_format: str
tone: str
knowledge_cutoff: Optional[str] = None
special_instructions: List[str] = None
class SystemPromptBuilder:
"""
Production-ready system prompt builder with templates and validation.
Usage:
builder = SystemPromptBuilder()
config = SystemPromptConfig(
role="customer service agent",
domain="fitness studio booking",
constraints=["never disclose pricing without verification"],
output_format="structured JSON",
tone="friendly and professional"
)
prompt = builder.build(config)
"""
ROLE_TEMPLATES = {
"customer_service": (
"You are a helpful customer service representative for {domain}. "
"Your goal is to resolve customer inquiries efficiently while "
"maintaining a positive brand experience."
),
"technical_assistant": (
"You are an expert technical assistant specializing in {domain}. "
"Provide accurate, detailed answers with code examples when appropriate."
),
"content_creator": (
"You are a creative content writer focused on {domain}. "
"Generate engaging, original content that resonates with the target audience."
),
"data_analyst": (
"You are a data analyst expert in {domain}. "
"Provide insights backed by logical reasoning and statistical understanding."
)
}
def build(self, config: SystemPromptConfig) -> str:
"""
Build a comprehensive system prompt from configuration.
Args:
config: SystemPromptConfig with role, constraints, format specs
Returns:
Complete system prompt string ready for API calls
"""
sections = []
# Role definition
role_template = self._get_role_template(config.role)
sections.append(role_template.format(domain=config.domain))
# Constraints section
if config.constraints:
sections.append("\n**Critical Constraints:**")
for i, constraint in enumerate(config.constraints, 1):
sections.append(f"{i}. {constraint}")
# Output format specification
sections.append(f"\n**Output Format:**\n{config.output_format}")
# Tone guidance
sections.append(f"\n**Communication Tone:**\n{config.tone}")
# Knowledge cutoff (if specified)
if config.knowledge_cutoff:
sections.append(
f"\n**Knowledge Cutoff:**\nYour training data extends to "
f"{config.knowledge_cutoff}. For recent events, acknowledge "
f"this limitation and focus on general principles."
)
# Special instructions
if config.special_instructions:
sections.append("\n**Special Instructions:**")
for instruction in config.special_instructions:
sections.append(f"- {instruction}")
return "\n".join(sections)
def _get_role_template(self, role: str) -> str:
"""Get the base template for a given role type."""
for key, template in self.ROLE_TEMPLATES.items():
if key in role.lower():
return template
# Default template if no match
return "You are a {domain} specialist. Provide helpful and accurate assistance."
def validate(self, prompt: str) -> Dict[str, bool]:
"""
Validate a system prompt for common issues.
Returns:
Dictionary with validation results
"""
validations = {
"has_role_definition": len(prompt) > 50,
"has_constraints": "constraint" in prompt.lower(),
"has_output_format": "format" in prompt.lower(),
"reasonable_length": 100 <= len(prompt) <= 2000,
"no_conflicting_instructions": not self._has_conflicts(prompt)
}
return validations
def _has_conflicts(self, prompt: str) -> bool:
"""Check for conflicting instructions."""
conflicts = [
("always" in prompt.lower() and "never" in prompt.lower()),
("must" in prompt.lower() and "optional" in prompt.lower())
]
return any(conflicts)
# Production example
if __name__ == "__main__":
builder = SystemPromptBuilder()
config = SystemPromptConfig(
role="customer_service",
domain="fitness studio booking and class management",
constraints=[
"Never disclose pricing without verifying user membership status",
"Always confirm class availability before suggesting booking",
"Redirect billing issues to human support",
"Maintain user privacy - never share personal information"
],
output_format=(
"Respond in JSON format:\n"
"{\n"
' "message": "user-facing response",\n'
' "action": "suggested_action",\n'
' "requires_human": boolean\n'
"}"
),
tone="Friendly, empathetic, and solution-focused. Use warm language while maintaining professionalism.",
knowledge_cutoff="April 2024",
special_instructions=[
"Use the customer's name when known",
"Offer alternative solutions when primary request cannot be fulfilled",
"End each response with a clear next step"
]
)
system_prompt = builder.build(config)
print("Generated System Prompt:")
print("=" * 60)
print(system_prompt)
print("\n" + "=" * 60)
# Validate the prompt
validation_results = builder.validate(system_prompt)
print("\nValidation Results:")
for check, passed in validation_results.items():
status = "✓" if passed else "✗"
print(f"{status} {check}")
This system prompt builder provides structure and consistency across your application. The validation methods catch common mistakes before they reach production.
Few-Shot Learning: Teaching Through Examples
Few-shot learning dramatically improves output quality by showing the model exactly what you expect. Instead of describing desired behavior, you demonstrate it with 2-5 carefully selected examples. The model recognizes patterns and replicates the structure, tone, and reasoning approach.
Here's a production few-shot formatter with example selection strategies:
from typing import List, Tuple, Optional
from dataclasses import dataclass
import random
@dataclass
class FewShotExample:
"""Single few-shot example with input and expected output."""
input: str
output: str
context: Optional[str] = None
explanation: Optional[str] = None
class FewShotFormatter:
"""
Few-shot prompt formatter with example selection and diversity optimization.
Production features:
- Automatic example diversity checking
- Token-aware example selection
- Format consistency validation
- Similarity-based example retrieval
"""
def __init__(self, max_examples: int = 5, max_tokens: int = 1500):
self.max_examples = max_examples
self.max_tokens = max_tokens
self.example_library: List[FewShotExample] = []
def add_example(self, example: FewShotExample) -> None:
"""Add example to the library."""
self.example_library.append(example)
def format_prompt(
self,
task_description: str,
user_input: str,
num_examples: Optional[int] = None,
diverse: bool = True
) -> str:
"""
Format a complete few-shot prompt.
Args:
task_description: What the model should do
user_input: The actual user query to respond to
num_examples: Number of examples to include (default: self.max_examples)
diverse: Whether to maximize example diversity
Returns:
Complete prompt string with task, examples, and user input
"""
num_examples = num_examples or self.max_examples
# Select examples
if diverse:
selected = self._select_diverse_examples(num_examples)
else:
selected = self.example_library[:num_examples]
# Build prompt sections
sections = [
f"# Task\n{task_description}\n",
"# Examples\n"
]
for i, example in enumerate(selected, 1):
sections.append(f"## Example {i}")
if example.context:
sections.append(f"Context: {example.context}")
sections.append(f"Input: {example.input}")
sections.append(f"Output: {example.output}")
if example.explanation:
sections.append(f"Explanation: {example.explanation}")
sections.append("") # Blank line between examples
sections.append(f"# Your Turn\nInput: {user_input}\nOutput:")
return "\n".join(sections)
def _select_diverse_examples(self, num_examples: int) -> List[FewShotExample]:
"""
Select diverse examples using simple keyword-based diversity.
In production, you'd use embedding-based similarity.
"""
if len(self.example_library) <= num_examples:
return self.example_library
selected = [self.example_library[0]] # Always include first example
remaining = self.example_library[1:]
while len(selected) < num_examples and remaining:
# Find example most different from selected ones
best_candidate = None
max_diversity = -1
for candidate in remaining:
diversity = self._calculate_diversity(candidate, selected)
if diversity > max_diversity:
max_diversity = diversity
best_candidate = candidate
if best_candidate:
selected.append(best_candidate)
remaining.remove(best_candidate)
return selected
def _calculate_diversity(
self,
candidate: FewShotExample,
selected: List[FewShotExample]
) -> float:
"""
Calculate diversity score for a candidate example.
Simple implementation using keyword overlap.
Production systems should use embedding cosine distance.
"""
candidate_words = set(candidate.input.lower().split())
total_overlap = 0
for example in selected:
example_words = set(example.input.lower().split())
overlap = len(candidate_words & example_words)
total_overlap += overlap
# Lower overlap = higher diversity
return 1.0 / (1.0 + total_overlap)
def estimate_tokens(self, prompt: str) -> int:
"""
Rough token estimation (4 chars ≈ 1 token).
Production should use tiktoken library for accurate counts.
"""
return len(prompt) // 4
def validate_examples(self) -> Dict[str, bool]:
"""Validate example library for common issues."""
return {
"has_examples": len(self.example_library) > 0,
"sufficient_examples": len(self.example_library) >= 3,
"consistent_format": self._check_format_consistency(),
"within_token_limit": all(
self.estimate_tokens(ex.input + ex.output) < 400
for ex in self.example_library
)
}
def _check_format_consistency(self) -> bool:
"""Check if all examples follow similar format."""
if len(self.example_library) < 2:
return True
# Check if outputs have similar length (within 50% variance)
output_lengths = [len(ex.output) for ex in self.example_library]
avg_length = sum(output_lengths) / len(output_lengths)
for length in output_lengths:
if abs(length - avg_length) / avg_length > 0.5:
return False
return True
# Production example
if __name__ == "__main__":
formatter = FewShotFormatter(max_examples=3)
# Add training examples
formatter.add_example(FewShotExample(
input="I want to cancel my class reservation for tomorrow",
output=json.dumps({
"action": "cancel_reservation",
"message": "I can help you cancel your class reservation. Which class were you registered for tomorrow?",
"requires_human": False
}),
explanation="Acknowledge request, ask for clarification"
))
formatter.add_example(FewShotExample(
input="Why was I charged twice this month?",
output=json.dumps({
"action": "escalate_billing",
"message": "I apologize for the billing confusion. Let me connect you with our billing specialist who can review your account and resolve this immediately.",
"requires_human": True
}),
explanation="Billing issues require human escalation"
))
formatter.add_example(FewShotExample(
input="What yoga classes do you offer on weekends?",
output=json.dumps({
"action": "provide_schedule",
"message": "We offer three yoga classes on weekends: Vinyasa Flow (Saturday 9am), Restorative Yoga (Saturday 4pm), and Power Yoga (Sunday 10am). Would you like to book any of these?",
"requires_human": False
}),
explanation="Provide specific information with next-step CTA"
))
# Generate prompt for new query
prompt = formatter.format_prompt(
task_description=(
"You are a fitness studio assistant. Respond to customer inquiries "
"with appropriate actions and messages. Always output JSON format."
),
user_input="Can I bring a friend to my personal training session?"
)
print("Few-Shot Prompt:")
print("=" * 60)
print(prompt)
print("\n" + "=" * 60)
# Validate examples
validation = formatter.validate_examples()
print("\nExample Library Validation:")
for check, passed in validation.items():
status = "✓" if passed else "✗"
print(f"{status} {check}")
Few-shot learning is particularly powerful for tasks requiring specific output formats, domain-specific reasoning, or consistent tone. The diversity selection ensures examples cover different query types.
For more advanced training techniques, see our guide on fine-tuning GPT models for ChatGPT applications.
Chain-of-Thought: Improving Reasoning Quality
Chain-of-thought (CoT) prompting instructs the model to show its reasoning process before providing an answer. This technique dramatically improves accuracy on complex tasks by forcing the model to break down problems into logical steps.
Instead of jumping to conclusions, the model articulates intermediate steps, catches logical errors, and provides transparent reasoning users can verify.
Here's a production chain-of-thought prompter with self-correction:
from typing import Dict, List, Optional, Callable
from dataclasses import dataclass
from enum import Enum
class ReasoningStyle(Enum):
"""Different chain-of-thought reasoning approaches."""
STEP_BY_STEP = "step_by_step"
PROS_CONS = "pros_cons"
HYPOTHESIS_TESTING = "hypothesis_testing"
ANALYTICAL = "analytical"
@dataclass
class ChainOfThoughtConfig:
"""Configuration for chain-of-thought prompting."""
style: ReasoningStyle
require_confidence: bool = True
enable_self_correction: bool = True
max_reasoning_steps: int = 5
class ChainOfThoughtPrompter:
"""
Production chain-of-thought prompt generator.
Supports multiple reasoning styles, self-correction,
and confidence estimation.
"""
STYLE_TEMPLATES = {
ReasoningStyle.STEP_BY_STEP: (
"Let's approach this step-by-step:\n"
"1. First, identify what we know\n"
"2. Then, determine what we need to find\n"
"3. Next, apply relevant principles or rules\n"
"4. Finally, verify the conclusion\n\n"
"Reasoning:"
),
ReasoningStyle.PROS_CONS: (
"Let's analyze this by weighing options:\n"
"1. List all viable options\n"
"2. For each option, identify pros and cons\n"
"3. Evaluate which option best fits the criteria\n"
"4. Make a recommendation with justification\n\n"
"Analysis:"
),
ReasoningStyle.HYPOTHESIS_TESTING: (
"Let's test our assumptions:\n"
"1. State initial hypothesis\n"
"2. Identify evidence that supports or contradicts it\n"
"3. Consider alternative explanations\n"
"4. Draw conclusion based on strongest evidence\n\n"
"Hypothesis Testing:"
),
ReasoningStyle.ANALYTICAL: (
"Let's break this down analytically:\n"
"1. Define key concepts and terms\n"
"2. Examine relationships between components\n"
"3. Apply logical deduction\n"
"4. Synthesize findings into conclusion\n\n"
"Analysis:"
)
}
def __init__(self, config: ChainOfThoughtConfig):
self.config = config
def build_prompt(
self,
query: str,
domain_context: Optional[str] = None,
constraints: Optional[List[str]] = None
) -> str:
"""
Build complete chain-of-thought prompt.
Args:
query: The question or task to reason about
domain_context: Relevant background information
constraints: Specific requirements or limitations
Returns:
Complete CoT prompt string
"""
sections = []
# Add domain context if provided
if domain_context:
sections.append(f"**Context:**\n{domain_context}\n")
# Add constraints if provided
if constraints:
sections.append("**Constraints:**")
for constraint in constraints:
sections.append(f"- {constraint}")
sections.append("")
# Add the query
sections.append(f"**Question:**\n{query}\n")
# Add reasoning instructions based on style
reasoning_template = self.STYLE_TEMPLATES[self.config.style]
sections.append(reasoning_template)
# Add self-correction instruction
if self.config.enable_self_correction:
sections.append(
"\n**Self-Check:**\n"
"Before finalizing your answer, review your reasoning:\n"
"- Are there any logical gaps or unsupported assumptions?\n"
"- Have you considered alternative perspectives?\n"
"- Does the conclusion follow from the reasoning?\n"
)
# Add confidence requirement
if self.config.require_confidence:
sections.append(
"\n**Confidence Assessment:**\n"
"Rate your confidence in this answer (Low/Medium/High) and explain why."
)
# Add final answer instruction
sections.append(
"\n**Final Answer:**\n"
"Based on the reasoning above, provide your final answer."
)
return "\n".join(sections)
def parse_response(self, response: str) -> Dict[str, str]:
"""
Parse a chain-of-thought response into components.
Returns:
Dictionary with reasoning, confidence, and final_answer
"""
components = {
"reasoning": "",
"confidence": "",
"final_answer": ""
}
# Simple parsing based on section headers
sections = response.split("**")
current_section = None
for i, section in enumerate(sections):
section_lower = section.lower().strip()
if "reasoning" in section_lower or "analysis" in section_lower:
current_section = "reasoning"
elif "confidence" in section_lower:
current_section = "confidence"
elif "final answer" in section_lower:
current_section = "final_answer"
elif current_section and section.strip():
components[current_section] += section.strip() + "\n"
return {k: v.strip() for k, v in components.items()}
def validate_reasoning(
self,
response: str,
validators: Optional[List[Callable[[str], bool]]] = None
) -> Dict[str, bool]:
"""
Validate chain-of-thought response quality.
Args:
response: The model's response
validators: Optional custom validation functions
Returns:
Dictionary of validation results
"""
parsed = self.parse_response(response)
validations = {
"has_reasoning": len(parsed["reasoning"]) > 50,
"has_final_answer": len(parsed["final_answer"]) > 10,
"reasoning_has_steps": parsed["reasoning"].count("\n") >= 3,
"has_confidence": len(parsed["confidence"]) > 0 if self.config.require_confidence else True
}
# Run custom validators
if validators:
for i, validator in enumerate(validators):
validations[f"custom_validation_{i}"] = validator(response)
return validations
# Production example
if __name__ == "__main__":
config = ChainOfThoughtConfig(
style=ReasoningStyle.STEP_BY_STEP,
require_confidence=True,
enable_self_correction=True
)
prompter = ChainOfThoughtPrompter(config)
prompt = prompter.build_prompt(
query="Should our fitness studio offer a 24-hour access option?",
domain_context=(
"We're a boutique fitness studio with 500 members. "
"Current hours: 5am-10pm weekdays, 7am-8pm weekends. "
"Staffed during all hours. Monthly membership: $149."
),
constraints=[
"Must maintain current service quality",
"Budget for staffing changes: $5,000/month maximum",
"Member surveys show 23% interest in late-night access"
]
)
print("Chain-of-Thought Prompt:")
print("=" * 60)
print(prompt)
print("\n" + "=" * 60)
# Example response parsing (would come from API)
example_response = """
**Reasoning:**
Step 1: What we know
- 500 members, 23% interested in 24hr access (115 members)
- Current hours already extensive (17 hours weekdays, 13 hours weekends)
- All hours currently staffed
- Budget constraint: $5k/month max
Step 2: What we need to determine
- Cost of 24hr staffing vs. value to members
- Safety and liability implications
- Impact on current staff and culture
Step 3: Analysis
- Unstaffed 24hr access requires security system ($2k setup, $200/month)
- Liability insurance increase: ~$300/month
- Lost boutique experience (no instructors at night)
- Only 115 members expressed interest - may not justify cost
Step 4: Verification
- Total new costs: ~$500/month (minimal staffing needed)
- Would need ~4 new members at $149/month to break even
- Risk: Could alienate existing members who value staffed experience
**Self-Check:**
- Assumption: 23% interest translates to actual usage (may be lower)
- Alternative: Offer extended hours (5am-midnight) as trial first
- Conclusion follows: Low risk if implemented as unstaffed trial
**Confidence Assessment:**
Medium confidence. The financial case is clear, but uncertain about member retention impact and actual usage rates. Would recommend 3-month pilot program before full commitment.
**Final Answer:**
Recommend a 3-month pilot of unstaffed 24hr access using keycard entry. Install security system, update insurance, and survey users monthly. This tests demand without long-term commitment while keeping costs under budget.
"""
parsed = prompter.parse_response(example_response)
print("\nParsed Response Components:")
for component, content in parsed.items():
print(f"\n{component.upper()}:")
print(content[:200] + "..." if len(content) > 200 else content)
# Validate the response
validations = prompter.validate_reasoning(example_response)
print("\n" + "=" * 60)
print("Validation Results:")
for check, passed in validations.items():
status = "✓" if passed else "✗"
print(f"{status} {check}")
Chain-of-thought prompting is essential for complex decision-making tasks, multi-step problems, and scenarios requiring transparency. The self-correction mechanism significantly reduces errors.
Learn more about designing effective conversation flows in our guide on conversation design for ChatGPT applications.
Prompt Templates: Reusable Patterns and Version Control
Production ChatGPT applications need consistent, reusable prompts that can be updated without code changes. Prompt templates provide variable substitution, versioning, and centralized management.
Here's a production template engine with version control:
from typing import Dict, Any, Optional, List
from dataclasses import dataclass, field
from datetime import datetime
import json
import hashlib
@dataclass
class PromptTemplate:
"""Versioned prompt template with metadata."""
name: str
template: str
version: str
variables: List[str]
description: str
created_at: datetime = field(default_factory=datetime.now)
tags: List[str] = field(default_factory=list)
def get_hash(self) -> str:
"""Generate hash of template content for change detection."""
content = f"{self.template}{self.version}"
return hashlib.md5(content.encode()).hexdigest()
class PromptTemplateEngine:
"""
Production prompt template engine with versioning and validation.
Features:
- Variable substitution with validation
- Template versioning and rollback
- Change tracking and audit logs
- Template inheritance and composition
"""
def __init__(self):
self.templates: Dict[str, Dict[str, PromptTemplate]] = {}
self.active_versions: Dict[str, str] = {}
self.audit_log: List[Dict[str, Any]] = []
def register_template(
self,
template: PromptTemplate,
set_active: bool = True
) -> None:
"""
Register a new template version.
Args:
template: The PromptTemplate to register
set_active: Whether to set this as the active version
"""
if template.name not in self.templates:
self.templates[template.name] = {}
self.templates[template.name][template.version] = template
if set_active:
self.active_versions[template.name] = template.version
self._log_action("register", template.name, template.version)
def render(
self,
template_name: str,
variables: Dict[str, Any],
version: Optional[str] = None
) -> str:
"""
Render a template with variable substitution.
Args:
template_name: Name of the template to render
variables: Dictionary of variable values
version: Specific version (default: active version)
Returns:
Rendered prompt string
Raises:
ValueError: If template not found or required variables missing
"""
# Get template
if template_name not in self.templates:
raise ValueError(f"Template '{template_name}' not found")
version = version or self.active_versions.get(template_name)
if not version or version not in self.templates[template_name]:
raise ValueError(f"Version '{version}' not found for template '{template_name}'")
template = self.templates[template_name][version]
# Validate variables
missing_vars = set(template.variables) - set(variables.keys())
if missing_vars:
raise ValueError(f"Missing required variables: {missing_vars}")
# Render template
rendered = template.template
for var_name, var_value in variables.items():
placeholder = "{" + var_name + "}"
rendered = rendered.replace(placeholder, str(var_value))
self._log_action("render", template_name, version)
return rendered
def set_active_version(self, template_name: str, version: str) -> None:
"""Set the active version for a template."""
if template_name not in self.templates:
raise ValueError(f"Template '{template_name}' not found")
if version not in self.templates[template_name]:
raise ValueError(f"Version '{version}' not found")
self.active_versions[template_name] = version
self._log_action("set_active", template_name, version)
def list_versions(self, template_name: str) -> List[str]:
"""List all versions of a template."""
if template_name not in self.templates:
return []
return list(self.templates[template_name].keys())
def get_template(
self,
template_name: str,
version: Optional[str] = None
) -> Optional[PromptTemplate]:
"""Retrieve a specific template version."""
if template_name not in self.templates:
return None
version = version or self.active_versions.get(template_name)
return self.templates[template_name].get(version)
def compare_versions(
self,
template_name: str,
version1: str,
version2: str
) -> Dict[str, Any]:
"""
Compare two template versions.
Returns:
Dictionary with comparison results
"""
t1 = self.get_template(template_name, version1)
t2 = self.get_template(template_name, version2)
if not t1 or not t2:
raise ValueError("One or both versions not found")
return {
"template_changed": t1.template != t2.template,
"variables_added": list(set(t2.variables) - set(t1.variables)),
"variables_removed": list(set(t1.variables) - set(t2.variables)),
"hash_v1": t1.get_hash(),
"hash_v2": t2.get_hash(),
"created_at_v1": t1.created_at.isoformat(),
"created_at_v2": t2.created_at.isoformat()
}
def _log_action(self, action: str, template_name: str, version: str) -> None:
"""Log template actions for audit trail."""
self.audit_log.append({
"timestamp": datetime.now().isoformat(),
"action": action,
"template": template_name,
"version": version
})
def export_templates(self, filepath: str) -> None:
"""Export all templates to JSON file."""
export_data = {
"templates": {},
"active_versions": self.active_versions,
"export_timestamp": datetime.now().isoformat()
}
for name, versions in self.templates.items():
export_data["templates"][name] = {}
for version, template in versions.items():
export_data["templates"][name][version] = {
"template": template.template,
"version": template.version,
"variables": template.variables,
"description": template.description,
"created_at": template.created_at.isoformat(),
"tags": template.tags
}
with open(filepath, 'w') as f:
json.dump(export_data, f, indent=2)
# Production example
if __name__ == "__main__":
engine = PromptTemplateEngine()
# Register version 1.0 of customer service template
template_v1 = PromptTemplate(
name="customer_service_fitness",
template=(
"You are a {studio_name} customer service assistant.\n\n"
"Customer Query: {query}\n\n"
"Instructions:\n"
"- Respond in a {tone} manner\n"
"- Reference our {policy_area} policy when relevant\n"
"- Offer to escalate if needed\n\n"
"Response:"
),
version="1.0",
variables=["studio_name", "query", "tone", "policy_area"],
description="Basic customer service template",
tags=["customer_service", "fitness"]
)
engine.register_template(template_v1)
# Register improved version 1.1
template_v1_1 = PromptTemplate(
name="customer_service_fitness",
template=(
"You are a {studio_name} customer service assistant.\n\n"
"Customer: {customer_name}\n"
"Membership Tier: {membership_tier}\n"
"Query: {query}\n\n"
"Instructions:\n"
"- Use {customer_name}'s name in your response\n"
"- Respond in a {tone} manner\n"
"- Reference our {policy_area} policy when relevant\n"
"- Provide {membership_tier}-specific benefits when applicable\n"
"- Offer to escalate if needed\n\n"
"Response:"
),
version="1.1",
variables=["studio_name", "customer_name", "membership_tier", "query", "tone", "policy_area"],
description="Enhanced template with personalization",
tags=["customer_service", "fitness", "personalized"]
)
engine.register_template(template_v1_1, set_active=True)
# Render with version 1.1
rendered = engine.render(
"customer_service_fitness",
{
"studio_name": "Zenith Fitness",
"customer_name": "Sarah",
"membership_tier": "Premium",
"query": "Can I freeze my membership for vacation?",
"tone": "friendly and helpful",
"policy_area": "membership freeze"
}
)
print("Rendered Prompt (v1.1):")
print("=" * 60)
print(rendered)
print("\n" + "=" * 60)
# Compare versions
comparison = engine.compare_versions(
"customer_service_fitness",
"1.0",
"1.1"
)
print("\nVersion Comparison (1.0 vs 1.1):")
print(json.dumps(comparison, indent=2))
# List all versions
print(f"\nAvailable versions: {engine.list_versions('customer_service_fitness')}")
print(f"Active version: {engine.active_versions['customer_service_fitness']}")
Template engines enable rapid iteration without code deployments. Version control ensures you can roll back problematic changes instantly.
Evaluation: Measuring Prompt Quality
You can't improve what you don't measure. Production prompt engineering requires systematic evaluation to track quality, identify regressions, and validate improvements.
Here's a production prompt evaluator with A/B testing:
from typing import Dict, List, Callable, Any, Optional
from dataclasses import dataclass
from datetime import datetime
from enum import Enum
import statistics
class MetricType(Enum):
"""Types of evaluation metrics."""
ACCURACY = "accuracy"
RELEVANCE = "relevance"
COHERENCE = "coherence"
SAFETY = "safety"
LATENCY = "latency"
@dataclass
class EvaluationResult:
"""Result from a single evaluation."""
prompt_version: str
metric_type: MetricType
score: float
timestamp: datetime
metadata: Dict[str, Any]
class PromptEvaluator:
"""
Production prompt evaluation framework.
Supports multiple metrics, A/B testing, and statistical analysis.
"""
def __init__(self):
self.results: List[EvaluationResult] = []
self.evaluators: Dict[MetricType, Callable] = {}
def register_evaluator(
self,
metric_type: MetricType,
evaluator_func: Callable[[str, str], float]
) -> None:
"""
Register an evaluation function for a metric type.
Args:
metric_type: Type of metric
evaluator_func: Function that takes (prompt, response) and returns score 0-1
"""
self.evaluators[metric_type] = evaluator_func
def evaluate(
self,
prompt_version: str,
prompt: str,
response: str,
metrics: List[MetricType],
metadata: Optional[Dict[str, Any]] = None
) -> Dict[MetricType, float]:
"""
Evaluate a prompt-response pair across multiple metrics.
Returns:
Dictionary mapping metric types to scores
"""
scores = {}
metadata = metadata or {}
for metric in metrics:
if metric not in self.evaluators:
continue
score = self.evaluatorsmetric
scores[metric] = score
# Record result
self.results.append(EvaluationResult(
prompt_version=prompt_version,
metric_type=metric,
score=score,
timestamp=datetime.now(),
metadata=metadata
))
return scores
def compare_versions(
self,
version_a: str,
version_b: str,
metric: MetricType,
min_samples: int = 10
) -> Dict[str, Any]:
"""
Compare two prompt versions statistically.
Returns:
Dictionary with comparison statistics
"""
# Get scores for each version
scores_a = [
r.score for r in self.results
if r.prompt_version == version_a and r.metric_type == metric
]
scores_b = [
r.score for r in self.results
if r.prompt_version == version_b and r.metric_type == metric
]
if len(scores_a) < min_samples or len(scores_b) < min_samples:
return {
"error": f"Insufficient samples (need {min_samples}, have {len(scores_a)}/{len(scores_b)})"
}
# Calculate statistics
mean_a = statistics.mean(scores_a)
mean_b = statistics.mean(scores_b)
stdev_a = statistics.stdev(scores_a) if len(scores_a) > 1 else 0
stdev_b = statistics.stdev(scores_b) if len(scores_b) > 1 else 0
# Simple significance test (would use proper t-test in production)
difference = mean_b - mean_a
relative_improvement = (difference / mean_a * 100) if mean_a > 0 else 0
return {
"version_a": {
"mean": mean_a,
"stdev": stdev_a,
"samples": len(scores_a)
},
"version_b": {
"mean": mean_b,
"stdev": stdev_b,
"samples": len(scores_b)
},
"difference": difference,
"relative_improvement_percent": relative_improvement,
"winner": version_b if difference > 0 else version_a
}
def get_summary_stats(
self,
prompt_version: str,
metric: Optional[MetricType] = None
) -> Dict[str, Any]:
"""Get summary statistics for a prompt version."""
filtered_results = [
r for r in self.results
if r.prompt_version == prompt_version and (metric is None or r.metric_type == metric)
]
if not filtered_results:
return {"error": "No results found"}
scores = [r.score for r in filtered_results]
return {
"prompt_version": prompt_version,
"metric": metric.value if metric else "all",
"samples": len(scores),
"mean": statistics.mean(scores),
"median": statistics.median(scores),
"stdev": statistics.stdev(scores) if len(scores) > 1 else 0,
"min": min(scores),
"max": max(scores),
"first_evaluation": filtered_results[0].timestamp.isoformat(),
"last_evaluation": filtered_results[-1].timestamp.isoformat()
}
# Example evaluator functions
def accuracy_evaluator(prompt: str, response: str) -> float:
"""
Simplified accuracy evaluator.
Production version would use semantic similarity, fact-checking, etc.
"""
# Placeholder: Check if response is substantive
if len(response) < 50:
return 0.3
if len(response) > 500:
return 0.7
return 0.5
def relevance_evaluator(prompt: str, response: str) -> float:
"""
Simplified relevance evaluator.
Production version would use embedding similarity.
"""
# Placeholder: Check keyword overlap
prompt_words = set(prompt.lower().split())
response_words = set(response.lower().split())
overlap = len(prompt_words & response_words)
return min(overlap / 10, 1.0)
def safety_evaluator(prompt: str, response: str) -> float:
"""
Simplified safety evaluator.
Production version would use moderation API.
"""
# Placeholder: Check for unsafe keywords
unsafe_keywords = ["password", "credit card", "ssn", "hack"]
response_lower = response.lower()
for keyword in unsafe_keywords:
if keyword in response_lower:
return 0.0
return 1.0
# Production example
if __name__ == "__main__":
evaluator = PromptEvaluator()
# Register evaluators
evaluator.register_evaluator(MetricType.ACCURACY, accuracy_evaluator)
evaluator.register_evaluator(MetricType.RELEVANCE, relevance_evaluator)
evaluator.register_evaluator(MetricType.SAFETY, safety_evaluator)
# Simulate evaluations for version A
for i in range(15):
scores = evaluator.evaluate(
prompt_version="v1.0",
prompt="What are your business hours?",
response=f"We're open Monday-Friday 9am-5pm. Sample response {i}.",
metrics=[MetricType.ACCURACY, MetricType.RELEVANCE, MetricType.SAFETY]
)
# Simulate evaluations for version B (improved)
for i in range(15):
scores = evaluator.evaluate(
prompt_version="v1.1",
prompt="What are your business hours?",
response=f"Our business hours are Monday-Friday 9am-5pm and Saturday 10am-3pm. We're closed Sundays. Response {i}.",
metrics=[MetricType.ACCURACY, MetricType.RELEVANCE, MetricType.SAFETY]
)
# Compare versions
comparison = evaluator.compare_versions(
version_a="v1.0",
version_b="v1.1",
metric=MetricType.ACCURACY
)
print("A/B Test Results (Accuracy):")
print("=" * 60)
print(json.dumps(comparison, indent=2))
print("\n" + "=" * 60)
print("\nSummary Statistics (v1.1):")
summary = evaluator.get_summary_stats("v1.1", MetricType.ACCURACY)
print(json.dumps(summary, indent=2))
Systematic evaluation prevents prompt regressions and provides data-driven confidence in improvements. A/B testing ensures changes actually improve user experience.
For more on model evaluation strategies, see our guide on model selection and evaluation for ChatGPT applications.
Bringing It All Together: Production Prompt Engineering
Professional prompt engineering combines all these techniques into a cohesive workflow:
- Define system prompts that establish clear roles, constraints, and output formats
- Use few-shot examples to demonstrate desired behavior patterns
- Apply chain-of-thought for complex reasoning tasks requiring transparency
- Manage prompts as templates with versioning and centralized control
- Evaluate continuously using metrics and A/B testing to drive improvements
The code examples in this guide are production-ready starting points. Adapt them to your domain, add domain-specific evaluators, and build a prompt library that evolves with your application.
Remember: prompt engineering is iterative. Start with basic prompts, measure performance, identify failure modes, and refine systematically. The difference between amateur and professional ChatGPT apps lies in this systematic approach to prompt quality.
Build Better ChatGPT Apps with MakeAIHQ
Ready to apply these prompt engineering techniques to your own ChatGPT application? MakeAIHQ provides a no-code platform specifically designed for building, testing, and deploying professional ChatGPT apps.
Our AI Conversational Editor lets you design sophisticated prompts visually, test them in real-time, and deploy to the ChatGPT App Store in 48 hours. No coding required—just bring your domain expertise and let our platform handle the technical complexity.
Start building your ChatGPT app today with a free trial. Join hundreds of businesses already reaching 800 million ChatGPT users.
Related Resources:
- Complete Guide to Building ChatGPT Applications (Pillar Guide)
- Fine-Tuning GPT Models for ChatGPT Apps
- Model Selection and Evaluation for ChatGPT Apps
- Conversation Design Principles for ChatGPT
- ChatGPT App Store Submission Guide
External Resources: