Job Description
Analyze data, generate insights, and support development and evaluation of AI/ML and generative AI solutions, bridging traditional data analysis with emerging AI capabilities.
Responsibilities:
Analyze structured and unstructured data for trends, patterns, and actionable insights
Evaluate AI/ML and generative AI outputs (accuracy, quality, bias, relevance)
Build dashboards and reports to track AI system performance and business KPIs
Define metrics for model evaluation and monitoring with data scientists and AI engineers
Write and optimize SQL; perform EDA to support model development
Label, curate, and validate datasets for training and fine-tuning
Translate business requirements into analytical frameworks and present findings
Identify data quality issues and work with engineering to resolve them
Required:
5+ years in data analysis, BI, or related analytics roles
Strong SQL; Python (pandas, NumPy) or R
BI tools (Tableau, Power BI, Looker or similar)
AI/ML concepts, with exposure to evaluating model outputs or working with ML teams
Strong statistical analysis and communication skills
Large datasets in cloud warehouses (BigQuery, Snowflake, Redshift)
Preferred: Evaluating GenAI/LLM outputs (prompt quality, hallucination detection, response scoring), GCP tools (BigQuery, Looker, Vertex AI), A/B testing, data governance/quality, scikit-learn.
Good to have: GCP, Vertex AI, data modeling, ETL/ELT, data quality, data visualization, prompt engineering, RAG, Cloud Storage, Dataflow.