Testing in the Lifecycle - Session 2A

t significant faults found firstfaults prevented, not built inno additional effort, re-schedule test designchanging requirements caused by test designEarly test design helps to build quality,stops fault multiplication6Experience report: Phase 1Phase 1: Plan2 mo2 modevtesttest150 faults1st mo.50 faultsusersnothappyQualityfraught, lots of dev overtimeActual"has to go in"but didn't workfraught, lots of dev overtimeActual"has to go in"but didn't work7Experience report: Phase 2Phase 2: Plan2 mo6 wksdevtesttest50 faults1st mo.0 faultshappyusers!Qualitysmooth, not much for dev to doActualacc test: fullweek (vs half day)on time8VV&TVerificationthe process of evaluating a system or component to determine whether the products of the given development phase satisfy the conditions imposed at the start of that phase [BS 7925-1]Validationdetermination of the correctness of the products of software development with respect to the user needs and requirements [BS 7925-1]Testingthe process of exercising software to verify that it satisfies specified requirements and to detect faults9Verification, Validation and TestingVerificationValidationTestingAny10V-model exerciseThe V Model 	 - 	 ExerciseDSFDBuildComponentsBuildUnitsTDBuildSystemCodeBuildAssemblageVDSystemTestIntegrationTestReview FDReview TDTUTFUTReview DSReview VDAssembly TestExceptions:Conversion TestFOS: DN/Gldn11How would you test this spec?A computer program plays chess with one user. It displays the board and the pieces on the screen. Moves are made by dragging pieces.12“Testing is expensive”Compared to what?What is the cost of NOT testing, or of faults missed that should have been found in test?Cost to fix faults escalates the later the fault is foundPoor quality software costs more to useusers take more time to understand what to dousers make more mistakes in using itmorale suffers => lower productivityDo you know what it costs your organisation?13What do software faults cost?Have you ever accidentally destroyed a PC?knocked it off your desk?poured coffee into the hard disc drive?dropped it out of a 2nd storey window?How would you feel?How much would it cost?14Hypothetical Cost - 1(Loaded Salary cost: £50/hr)Fault Cost Developer User£700	 £50- detect ( .5 hr)	£25- report ( .5 hr)	£25- receive & process (1 hr)	 £50- assign & bkgnd (4 hrs)	£200- debug ( .5 hr)	 £25- test fault fix ( .5 hr)	 £25- regression test (8 hrs)	£40015Hypothetical Cost - 2Fault Cost Developer User	 £700	£50- update doc'n, CM (2 hrs)	 £100- update code library (1 hr)	 £50- inform users (1 hr)	 £50- admin(10% = 2 hrs)	 £100Total (20 hrs)	£100016Hypothetical Cost - 3Fault Cost Developer User	 £1000	 £50(suppose affects only 5 users)- work x 2, 1 wk	£4000- fix data (1 day)	 £350- pay for fix (3 days maint)	 £750- regr test & sign off (2 days)	 £700- update doc'n / inform (1 day)	 £350- double check + 12% 5 wks	£5000- admin (+7.5%)	 £800Totals	 £1000 £1200017Cost of fixing faultsReqUseDesTest110100010018How expensive for you?Do your own calculationcalculate cost of testingpeople’s time, machines, toolscalculate cost to fix faults found in testingcalculate cost to fix faults missed by testingEstimate if no data availableyour figures will be the best your company has!(10 minutes)19OverviewModels for testing, economics of testingHigh level test planningComponent TestingIntegration testing in the smallSystem testing (non-functional and functional)Integration testing in the largeAcceptance testing Maintenance testing20(Before planning for a set of tests)set organisational test strategyidentify people to be involved (sponsors, testers, QA, development, support, et al.)examine the requirements or functional specifications (test basis)set up the test organisation and infrastructuredefining test deliverables & reporting structureSee: Structured Testing, an introduction to TMap®, Pol & van Veenendaal, 199821High level test planningWhat is the purpose of a high level test plan?Who does it communicate to?Why is it a good idea to have one?What information should be in a high level test plan?What is your standard for contents of a test plan?Have you ever forgotten something important?What is not included in a test plan?22Test Plan 11 Test Plan Identifier2 Introductionsoftware items and features to be testedreferences to project authorisation, project plan, QA plan, CM plan, relevant policies & standards3 Test itemstest items including version/revision levelhow transmitted (net, disc, CD, etc.)references to software documentationSource: ANSI/IEEE Std 829-1998, Test Documentation23Test Plan 24 Features to be testedidentify test design specification / techniques5 Features not to be testedreasons for exclusion24Test Plan 36 Approachactivities, techniques and toolsdetailed enough to estimatespecify degree of comprehensiveness (e.g. coverage) and other completion criteria (e.g. faults)identify constraints (environment, staff, deadlines)7 Item Pass/Fail Criteria8 Suspension criteria and resumption criteriafor all or parts of testing activitieswhich activities must be repeated on resumption25Test Plan 49 Test DeliverablesTest planTest design specificationTest case specificationTest procedure specificationTest item transmittal reportsTest logsTest incident reportsTest summary reports26Test Plan 510 Testing tasksincluding inter-task dependencies & special skills11 Environmentphysical, hardware, software, toolsmode of usage, security, office space12 Responsibilitiesto manage, design, prepare, execute, witness, check, resolve issues, providing environment, providing the software to test27Test Plan 613 Staffing and Training Needs14 Scheduletest milestones in project scheduleitem transmittal milestonesadditional test milestones (environment ready)what resources are needed when15 Risks and Contingenciescontingency plan for each identified risk16 Approvalsnames and when approved28OverviewModels for testing, economics of testingHigh level test planningComponent TestingIntegration testing in the smallSystem testing (non-functional and functional)Integration testing in the largeAcceptance testing Maintenance testing29Component testing lowest leveltested in isolationmost thorough look at detailerror handlinginterfacesusually done by programmeralso known as unit, module, program testing30Component test strategy 1specify test design techniques and rationalefrom Section 3 of the standard*specify criteria for test completion and rationalefrom Section 4 of the standarddocument the degree of independence for test designcomponent author, another person, from different section, from different organisation, non-human* Source: BS 7925-2, Software Component Testing Standard31Component test strategy 2component integration and environmentisolation, top-down, bottom-up, or mixturehardware and softwaredocument test process and activitiesincluding inputs and outputs of each activityaffected activities are repeated after any fault fixes or changesproject component test plandependencies between component tests32Component Test Document HierarchySource: BS 7925-2, Software Component Testing Standard, Annex AComponentTest StrategyProjectComponentTest PlanComponentTestSpecificationComponentTest PlanComponentTest Report33Component test processChecking forComponentTest CompletionComponentTest PlanningComponentTest SpecificationComponentTest ExecutionComponentTest RecordingBEGINEND34Component test processComponentTest PlanningComponentTest SpecificationComponentTest ExecutionComponentTest RecordingChecking forComponentTest CompletionBEGINENDComponent test planning- how the test strategy and project test plan apply to the component under test- any exceptions to the strategy- all software the component will interact with (e.g. stubs and drivers35Component test processComponentTest PlanningComponentTest SpecificationComponentTest ExecutionComponentTest RecordingChecking forComponentTest CompletionBEGINENDComponent test specification- test cases are designed using the test case design techniques specified in the test plan (Section 3)- Test case: objective initial state of component input expected outcome- test cases should be repeatable36Component test processComponentTest PlanningComponentTest SpecificationComponentTest ExecutionComponentTest RecordingChecking forComponentTest CompletionBEGINENDComponent test execution- each test case is executed- standard does not specify whether executed manually or using a test execution tool37Component test processComponentTest PlanningComponentTest SpecificationComponentTest ExecutionComponentTest RecordingChecking forComponentTest CompletionBEGINENDComponent test recording- identities & versions of component, test specification- actual outcome recorded & compared to expected outcome- discrepancies logged- repeat test activities to establish removal of the discrepancy (fault in test or verify fix)- record coverage levels achieved for test completion criteria specified in test planSufficient to show test activities carried out38Component test processComponentTest PlanningComponentTest SpecificationComponentTest ExecutionComponentTest RecordingChecking forComponentTest CompletionBEGINENDChecking for component test completion- check test records against specified test completion criteria- if not met, repeat test activities- may need to repeat test specification to design test cases to meet completion criteria (e.g. white box)39Test design techniques“Black box”Equivalence partitioningBoundary value analysisState transition testingCause-effect graphingSyntax testingRandom testingHow to specify other techniques“White box”Statement testingBranch / Decision testingData flow testingBranch condition testingBranch condition combination testingModified condition decision testingLCSAJ testing= Yes= NoAlso a measurement technique?40Summary: Key PointsV-model shows test levels, early test designHigh level test planningComponent testing using the standard41Thank YouAiTi Educationpublished by www.aiti.edu.vn@aiti_aptechaiti.edu.vn42