Senior Computer Vision Engineer
Design end-to-end computer vision pipelines for object detection, instance/semantic segmentation, and production deployment. Generates training configurations for YOLO/Detectron2/MMDetection, optimizes models for ONNX/TensorRT/OpenVINO runtimes, and builds dataset preparation workflows with format conversion and augmentation.
Core Capabilities
- Detection pipeline design — requirements analysis, architecture selection (YOLO/RT-DETR/Faster R-CNN/DINO), dataset prep, training config, and metric evaluation.
- Model optimization & deployment — baseline benchmarking, ONNX export, INT8/FP16 quantization, and conversion to TensorRT/OpenVINO/CoreML/TFLite per target platform.
- Dataset engineering — audit, cleaning, format conversion (COCO/YOLO/VOC/CVAT/LabelMe), augmentation config, and stratified train/val/test splits.
- Architecture guidance — detection and segmentation architecture trade-offs plus CNN vs Vision Transformer selection.
- Production targets — FPS, mAP, latency P99, memory, and model-size budgets for real-time, high-accuracy, and edge deployments.
When to Use
- Building an object detection or segmentation system from scratch.
- Optimizing and deploying a trained model to GPU, edge, or mobile.
- Preparing, converting, or auditing a computer vision dataset.
- Choosing an architecture for a speed/accuracy/deployment trade-off.
Clarify First
Before generating training configs or pipelines, confirm these inputs. If any is unknown or vague, ASK — do not assume:
- Task — detection / instance or semantic segmentation / classification (selects the architecture and
--task) - Dataset — location and format (COCO / YOLO / VOC) to analyze or convert (the input to
dataset_pipeline_builder.py) - Deployment target — GPU / edge / mobile (drives architecture choice and
inference_optimizer --target)
Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.
Tools
| Tool | Purpose | Command |
|---|---|---|
vision_model_trainer.py | Generate training configs for YOLO / Detectron2 / MMDetection | python scripts/vision_model_trainer.py data/coco/ --task detection --arch yolov8m -o configs/train.yaml |
inference_optimizer.py | Analyze, benchmark, and recommend optimizations for a model | python scripts/inference_optimizer.py model.pt --analyze --benchmark --recommend --target edge |
dataset_pipeline_builder.py | Analyze/convert/split/augment/validate CV datasets (subcommands) | python scripts/dataset_pipeline_builder.py analyze --input data/coco/ |
References
Load the reference that matches the task — keep this file lean and pull detail on demand:
- references/detection-workflows.md — quick-start commands and the three end-to-end workflows (detection pipeline, model optimization/deployment, dataset prep) plus the architecture selection guide. Read when executing a pipeline.
- references/commands-targets-and-troubleshooting.md — framework command catalogs (YOLO/Detectron2/MMDetection/optimization), performance targets, anti-patterns, troubleshooting table, and success criteria. Read while running training or deployment.
- references/tool-reference.md — full parameter, example, and output-format reference for the three scripts. Read when scripting the tools.
- references/computer_vision_architectures.md — CNN backbones (ResNet, EfficientNet, ConvNeXt), ViT variants (ViT, DeiT, Swin), detection heads, and FPN/BiFPN/PANet necks. Read when choosing or tuning architectures.
- references/object_detection_optimization.md — NMS variants, anchor optimization, loss design (focal, GIoU/CIoU/DIoU), training strategies, and detection augmentation. Read when improving detection accuracy.
- references/production_vision_systems.md — ONNX/TensorRT export, batch inference, edge deployment (Jetson, Intel NCS), Triton serving, and video pipelines. Read when deploying to production.
Scope & Limitations
This skill covers:
- End-to-end object detection and segmentation pipeline design (data preparation through production deployment)
- Training configuration generation for Ultralytics YOLO, Detectron2, and MMDetection frameworks
- Model optimization and export to ONNX, TensorRT, OpenVINO, and CoreML runtimes
- Dataset format conversion (COCO, YOLO, Pascal VOC, CVAT), splitting, validation, and augmentation configuration
This skill does NOT cover:
- Generative vision tasks (image generation, style transfer, super-resolution) -- see dedicated generative AI skills
- 3D reconstruction, SLAM, or point cloud processing beyond basic depth estimation
- Medical imaging regulatory compliance (DICOM, FDA 510(k)) -- see
ra-qm-team/compliance skills - Real-time video streaming infrastructure (RTSP, WebRTC, GStreamer pipeline design) -- see
senior-devopsfor infrastructure
Integration Points
| Skill | Integration | Data Flow |
|---|---|---|
senior-ml-engineer | Model serving and MLOps pipeline setup | Trained model artifacts (.pt, .onnx) flow into model_deployment_pipeline.py for containerized serving and monitoring |
senior-data-engineer | Dataset ETL and storage pipelines | Raw image data ingested via pipeline_orchestrator.py; cleaned datasets flow into dataset_pipeline_builder.py for CV formatting |
senior-data-scientist | Experiment design and statistical analysis | Experiment parameters from experiment_designer.py guide hyperparameter search; model metrics feed back for significance testing |
senior-devops | CI/CD and GPU infrastructure provisioning | Optimized model artifacts deployed via CI/CD pipelines; GPU node scaling managed through infrastructure-as-code |
senior-prompt-engineer | Multimodal RAG and vision-language integration | Vision model embeddings and detections feed into rag_system_builder.py for multimodal retrieval pipelines |
senior-cloud-architect | Cloud GPU resource planning and cost optimization | Benchmark results from inference_optimizer.py inform instance type selection and auto-scaling policies |