Frappe Data Migration Generator
Generate robust data migration scripts with validation, error handling, and progress tracking for importing data into Frappe.
When to Use This Skill
Claude should invoke this skill when:
- User wants to migrate data from legacy systems
- User needs to import large CSV/Excel files
- User mentions data migration, ETL, or data import
- User wants to transform data structures
- User needs bulk data operations
Capabilities
1. CSV Import Script
Production-Ready CSV Importer:
import csv
import frappe
from frappe.utils import flt, cint, getdate
def import_customers_from_csv(file_path):
"""Import customers with validation and error handling"""
success = []
errors = []
with open(file_path, 'r', encoding='utf-8-sig') as f:
reader = csv.DictReader(f)
for idx, row in enumerate(reader, start=2):
try:
# Validate required fields
if not row.get('Customer Name'):
raise ValueError('Customer name required')
# Transform data
customer = {
'doctype': 'Customer',
'customer_name': row['Customer Name'].strip(),
'customer_group': row.get('Customer Group', 'Commercial'),
'territory': row.get('Territory', 'All Territories'),
'email_id': row.get('Email', '').strip(),
'mobile_no': row.get('Phone', '').strip(),
'credit_limit': flt(row.get('Credit Limit', 0))
}
# Check duplicate
exists = frappe.db.exists('Customer',
{'customer_name': customer['customer_name']})
if exists:
# Update
doc = frappe.get_doc('Customer', exists)
doc.update(customer)
doc.save()
else:
# Insert
doc = frappe.get_doc(customer)
doc.insert()
success.append(row['Customer Name'])
# Commit every 100
if len(success) % 100 == 0:
frappe.db.commit()
print(f"Processed {len(success)} records")
except Exception as e:
errors.append({'row': idx, 'data': row, 'error': str(e)})
frappe.db.commit()
return {'success': success, 'errors': errors}
References
Frappe Data Import: