o h^2@sddlmZddlmZddlmZmZddlZddlZddl m Z e dde dZ e d Z e d Ze d Zejd e e ed dZejedGdddejZGdddejZGdddeZGdddeZdS))Type)BaseTool) BaseModelFieldN) load_dotenvz7/home/azureuser/microlearn/backend/prompt_opt_dspy/.env) dotenv_path AZURE_API_KEYAZURE_API_HOSTAZURE_DEPLOYMENT_IDAZURE_API_VERSIONzazure/Csqr-gpt-4o-minig)api_keyapi_base api_version temperature)lmc@sVeZdZUdZeZeed<eZ eed<eZ eed<ej ddZ eed<dS) AnswerEvaluationSignaturea You are a Response Evaluator. Assess user understanding of a question based on their answer. The user's company context is: Company Name : {Company_Name} Industry : {Industry} Company Size : {Company_Size} Business Model : {Business_Model} Top Use Cases : {Top_Use_Cases} Learning Focus : {Learning_Focus} Tech Stack : {Tech_stack} Compliance : {Compliance} Task: - Evaluate the given user answer against the provided question and context. - Identify correctness, completeness, and clarity. - Highlight strengths in the response. - Suggest improvements if necessary. - Keep the feedback concise and constructive. Expected Output: Structured evaluation covering: - **Understanding Level** (Poor / Fair / Good / Excellent) - **Strengths** (bullet points) - **Areas to Improve** (bullet points) company_contextquestionanswerzJEvaluation including understanding level, strengths, and improvement areas)desc evaluationN) __name__ __module__ __qualname____doc__dspy InputFieldrstr__annotations__rr OutputFieldrr r /home/azureuser/microlearn/backend/user_journey_with_openai/agentic_workflow/src/user_journey_service/tools/custom_answer_eval_tool.pyrs rcs$eZdZfddZddZZS)AnswerEvaluatorModulecs\ttt}tjddddddddg}d d d }tj|d }|j||d |_ dS)Na Company Name: FinSight Industry: Financial Services Company Size: 250 employees Business Model: B2B Top Use Cases: Financial modeling Learning Focus: Data Analysis Tech Stack: Excel, Tableau, SQL Server Compliance: SOX, PCI DSS z5What is the role of SQL in analyzing structured data?zlSQL helps analysts query databases, filter records, and summarize large datasets to support decision-making.a **Understanding Level**: Good **Strengths**: - Correctly explains SQL's role in querying and summarizing data. - Focuses on structured data analysis. **Areas to Improve**: - Could mention SQL's ability to join multiple tables for deeper insights. )rrrrrrrcs2t|ddpdgd}ttfdd|DS)z Simple metric: ensure output contains - Understanding Level - Strengths - Areas to Improve r)zUnderstanding Level StrengthszAreas to Improvec3s|]}|vVqdSNr ).0partoutputr r! qszLAnswerEvaluatorModule.__init__..evaluation_metric..)getattrintall)goldpredtracerequired_partsr r(r!evaluation_metrichsz9AnswerEvaluatorModule.__init__..evaluation_metric)metric)studenttrainsetr%) super__init__rPredictrExample with_inputsBootstrapFewShotcompile predictor)selfbase_predictorr5r2 optimizer __class__r r!r7Cs      zAnswerEvaluatorModule.__init__cCs|j|||dS)Nrrr)r=)r>rrrr r r!forwardzs zAnswerEvaluatorModule.forward)rrrr7rD __classcell__r r rAr!r"Bs 7r"c@seZdZUdZedddZeed<edddZeed<edddZ eed <edd dZ eed <edd dZ eed <edddZ eed<edddZ eed<edddZeed<edddZeed<edddZeed<dS)AnswerEvaluatorInputz&Input schema for AnswerEvaluationTool..zName of the company) description Company_NamezIndustry of the companyIndustryzSize of the company Company_SizezCompany's business modelBusiness_Modelz$Top use cases the company focuses on Top_Use_CaseszLearning focus for the userLearning_FocuszCompany's tech stack Tech_stackz Relevant compliance requirements CompliancezThe question being answeredrzThe user's provided answerrN)rrrrrrHrrrIrJrKrLrMrNrOrrr r r r!rFs rFc@sleZdZUdZeed<dZeed<eZe e ed<dededed ed ed ed ed edededefddZ dS)AnswerEvaluationToolzAnswer Evaluation ToolnamezEvaluates a user's answer to a given question, assessing correctness, completeness, and clarity, while providing constructive feedback.rG args_schemarHrIrJrKrLrMrNrOrrreturnc CsRd|d|d|d|d|d|d|d|d } t} | j| | | d } | jS) Nz Company Name: z Industry: z Company Size: z Business Model: z Top Use Cases: z Learning Focus: z Tech Stack: z Compliance: z rC)r"rDr)r>rHrIrJrKrLrMrNrOrrr evaluatorresultr r r!_runs0 zAnswerEvaluationTool._runN) rrrrQrrrGrFrRrrrVr r r r!rPs8      rP)typingr crewai.toolsrpydanticrrrosdotenvrgetenv azure_api_keyazure_api_hostazure_deployment_idazure_api_versionLMr configure SignaturerModuler"rFrPr r r r!s.        &A