Some standard content:
Shipbuilding Industry Standard of the People's Republic of China
FL0137
CB1360-2002
Computer Software Testing Procedure
Test specification of computer software2002-11-20 Issued
Issued by the Commission of Science, Technology and Industry for National Defense
2003-02-01 Implementation
1 Scope
2 Normative reference documents
3 Terms and definitions
4 General requirements
4.1 Test purpose
4.2 Test organization
4.3 Basic entry conditions
4.4 Pass criteria
4.5 Test Test Documents
4.6 Basic Test Workflow
4.7 Disposal of Tested Software
4.8 Basic Requirements for Testing
4.9 Test Records and Control
4.10 Handling of Test Deviations
5 Detailed Requirements,.
5.1 Test Workflow,
5.2 Software Unit Testing
5.3 Software Component Testing
5.4 Computer Software Configuration Item Testing
|5.5 System testing
5.6 Acceptance testing
Appendix A (Informative Appendix) Contents of software test plan Appendix B (Normative Appendix) Code review
Appendix C (Informative Appendix) Format of software test problem report form Appendix D
Format and content of software test report
(Informative Appendix)
Appendix E (Informative Appendix) Format of the entry and exit registration form for the tested software, Appendix F (Informative Appendix) Format of the report on corrective measures for deviations from regulations and procedures or test differences Appendix G (Informative Appendix) Example of test case design outline for configuration items of ship command and control system intelligence processing software Appendix H (Normative Appendix)
Appendix D (Normative Appendix)
Regression test
Strength test
Appendix K (Informative Appendix) Software criticality level CB1360-2002
CB1360-2002
This standard is proposed by China Shipbuilding Industry Corporation. Foreword
This standard is under the jurisdiction of China Shipbuilding Industry Comprehensive Technology and Economic Research Institute. The drafting unit of this standard is the 716th Institute of China Shipbuilding Industry Corporation. The main drafters of this standard are Sun Zhian, Wan Qin, Cai Shaocheng, Chen Mingzhu, Pan Guanhua, and Huang Keming. 1 Scope
Computer Software Testing Procedure
CB1360-2002
This standard specifies the basic requirements for the computer software testing process, including test purpose, test organization, entry conditions, pass criteria, test documents, test workflow, disposal of tested software, test records and control, and test deviation handling. At the same time, it specifies the detailed requirements for different types of tests in the software testing process. This standard applies to the following testing activities of computer software, especially software developed according to GJB2786: a)
Software unit testing;
Software component testing:
Computer software configuration item testing;
System testing;
Acceptance testing;
Code review:
Strength testing;
Regression testing.
This standard also applies to the software development of the contractor. 2 Normative referenced documents
The clauses in the following documents become the clauses of this standard through reference in this standard. For any dated referenced document, all subsequent amendments (excluding errata) or revisions are not applicable to this standard. However, the parties to the agreement based on this standard are encouraged to study whether to use the latest versions of these documents. For any undated referenced document, the latest version shall apply to this standard. GB/T11457 Software Engineering Terminology
GJB438A—1997 Weapon System Software Development Documentation GJB2434—1995 General Requirements for Military Software Testing and Evaluation GJB2725 General Requirements for Calibration Laboratories and Test Laboratories 3 Terms and Definitions
This standard adopts the following terms and definitions. For other terms and definitions, please refer to GB/T11457.3.1
Test entrusting party is an organization or individual that entrusts the computer software testing task to a third-party testing organization. The test entrusting party is usually the developer, user or user representative of the computer software.
Testing party testparty
A third-party software testing organization that accepts the test entrustment and organizes and implements the computer software testing task, or a testing organization independent of the software developer.
Computer software configuration item is a set of software designed for independent configuration management and that can meet the end-user's requirements for functions, etc. It is referred to as software configuration item in the text of this standard.
CB1360-2002
Computer software componentcomputersoftwarecomponentThe entity obtained by hierarchical decomposition of computer programs. It is a part of computer software configuration items with different properties. Computer software components can be further decomposed into other software components or software units. 3.5
Computer software unitcomputersoftwareunitThe part that can be tested independently in the design of computer software components. 3.6
Software featuresoftwarefeature
The collection of software functions, performance, etc. specified in software requirements specifications, etc. 3.7
Software testingsoftwaretesting
The technical operation of determining or verifying one or more characteristics of specified software (units, software components, configuration items, etc.) according to specific procedures.
Software testing environmentsoftwaretestenvironmentThe collection of a set of tools, firmware, hardware, etc. required for software testing. Tools include but are not limited to testing tools, such as simulation software, code analyzers, static analysis tools, etc., and may also include tools for software engineering environments. 3.9
Software evaluationsoftwareevaluation
Testing, evaluation and other activities to measure software quality. 4 General requirements
4.1 Test objectives
The test objectives of computer software are as follows:
a) To verify whether the design and implementation of computer software meet the requirements specified in the task book or contract and the software requirement specification (including interface requirement specification);
To verify whether the defects and error statistics in the computer software are within the specified range, and to evaluate the quality of the tested software;
Through testing, defects and errors in computer software are discovered or detected to improve software reliability; c)
To provide a basis for various verifications, confirmations and decisions on whether the software can be accepted or used during the software development process; e) To provide a basis for the technical status confirmation of software configuration items and the acceptance and delivery of software products. 4.2 Testing Organization
The testing organization for computer software shall meet the following requirements: The testing organization shall be equipped with quality management personnel, testers, verifiers, and evaluators in accordance with the requirements of GJB2725, and shall specify their responsibilities, authorities and their mutual relationships: After accepting the testing commission, the testing organization shall set up a software testing group, which shall generally be composed of professional testers. b) When necessary, peer experts, users of the software being tested or user representatives may be invited to participate; All personnel in the testing organization shall undergo professional training and be certified before they can take up their posts; c) The organization and personnel of the testing organization shall have autonomy to ensure the independence of all testing processes and the objectivity, impartiality and fairness of the test results:
The testing organization shall formulate operating procedures for the testing environment and testing tools, and the testing environment and testing tools shall be calibrated (verified) and tested regularly:
CB1360--2002
f) The testing organization shall formulate corresponding procedures to ensure the reproducibility of the testing process and test results, and ensure the inspectability and traceability of test cases, test documents, etc.:
The testing organization shall establish a complete archive system as required; g) The testing work subcontracting, external services or supply of the testing organization shall meet the requirements of GJB2725. h)
4.3 Basic entry conditions
The basic entry conditions for computer software testing are as follows: The requirements for test documents should meet 4.5: a
Before testing, the tested software must pass self-testing or technical status inspection, and the test client should provide a self-test report or technical status inspection report b)
Specified test environment and test tools;
The tester and the test client should complete the review and confirmation of the test preparation and reach a consensus on all ambiguities; d)
e) For test entry conditions that need to be specifically clarified, they should be proposed by the tester and the test client after consensus. 4.4 Passing criteria
The tester shall determine the software test passing criteria based on the analysis of software features, test features, tested features and untested features in accordance with the contract and agreement. Usually, the test pass criteria are a collection or a subset of the following clauses: a) The test tasks specified in the contract and agreement have been completed as required; the test items and test contents specified in the test plan have been completed and the specified test coverage and technical requirements have been met; b) There are reasonable explanations and correct and effective handling of problems or anomalies found in the test; c) A complete software test problem report form is formed for the problems found in the troubleshooting test, and these problems are corrected as required, and the regression test or reorganized test has passed; all the defects found and their impacts have been eliminated. When they cannot be eliminated in time, the risks of these defects to the operation of the software have been identified and confirmed:
All ambiguities and related issues in the test have been agreed upon by the tester and the test client: f)
For software that needs to be fixed and run, the test has been completed in the specified firmware environment: g)
The test input documents and test output documents (see 4.5) required in the test process are complete and have been included in the corresponding h)
configuration management:
The test analysis and evaluation have been completed, and a reasonable and fair judgment has been made on the test results, and the preparation of the test report has been completed; i)
Specially required test pass criteria should be proposed by the tester and the test client through negotiation, included in the corresponding terms of the contract, and j)
clearly stipulated and described in documents such as the test plan and test instructions. 4.5 Test Documents
4.5.1 Classification of Test Documents
Corresponding to the test activities and tasks in each stage of software testing, test documents can be divided into two categories: test input documents and test output documents.
The categories of test documents and the requirements for documents are as follows: a) Test contract or agreement: Before testing, the tester and the test client shall negotiate and sign in accordance with the provisions of the Contract Law, etc.; Requirement specification (including interface requirement specification): The test client shall provide it according to the contract or test requirements. The format of requirement specification is shown in 5.3 and 5.4 of GJB438A-1997; Design document (including interface design document): The test client shall provide it according to the contract or test requirements. The format of design document is shown in 5.5 and 5.6 of GJB438A-1997; Source code: The test client shall provide it according to the contract or test requirements; d)
Test plan: The tester shall prepare it in the test planning stage. The test plan shall describe the scope, methods, resources and progress of the test activities. It specifies the tested features, tasks to be completed, responsibilities of testers and risks related to this plan based on the software features. The format and preparation requirements of the test plan are shown in Appendix A: 3
CB13602002
Test description: It should be prepared by the tester as needed during the test design phase. The test description should include the following three types of documents: f)
Test design description: It should describe the test method, specify the characteristics included in the test design and its related tests, and also specify the test cases and test procedures required to complete the test: 2)
Test case description: It should describe the specific values used for input and the expected output results, and specify various restrictions on the test procedures when using specific test cases; Test procedure description: It specifies the procedures and methods required for running the system and executing the specified test cases to implement the relevant test design.
In actual use, these three types of documents are usually combined into one document and the test description. The format of the test description can be found in 5.10 of GJB438A-1997.
Code review plan: It should be prepared by the reviewer before the code review. The code review plan should specify the scope, method, progress and personnel arrangement of the code review. The typical catalog and preparation requirements of the code review plan are shown in B.5.1.1 in Appendix B; Code review form: It should be prepared by the reviewer before the code review. The code review form should specify the requirements, goals, methods and workflow of the code review. The basic code review form is shown in B.5.1.2 in Appendix B: Code review problem report form: When problems are found during the code review process, the code reviewer should fill it out. The format of the code review problem report form is shown in B.5.1.3 in Appendix B: Software test problem report form: When problems are found during the software testing process, the software tester should fill it out. The format of the software test problem report form is shown in Appendix C; Software test report: After the software testing is completed, the tester should prepare it according to the test process, test results, etc. The format and content of the test report are shown in Appendix D or 5.11.2 in GJB438A-1997; Test record (test log): The tester uses it to record the software test process, test data, test results, test processing, etc.
Selection of test documents
The selection of test input documents and test output documents at each stage of the software testing process is shown in Table 1. Table 1 List of documents required for testing
Test document category
Contract or agreement
Requirement specification
Design document
Source code
Test planbZxz.net
Test description
Code review plan
Code review form
Code review problem report
Code review report
Software test problem report
Test report
Test/review record
Code review
Unit test
Software test category
Soft component test
Configuration item test
Note: / indicates a required document: indicates an optional document or a document generated according to actual conditions: 4
System test
Indicates that no document is selected.
Acceptance test
4.5.2.2 The selection criteria for test documents are as follows: CB1360-2002
For the testing of smaller-scale software, the relevant contents of the test description can be described in the test plan without the need to prepare a separate test description a)
For the code review of smaller-scale software, when formulating the code review plan, the relevant contents of the code review form can be considered together without the need to prepare a separate code review form; the content and format of documents such as the test plan, test description, test report, code review plan, code review form, code review report, etc. can also be prepared in the format specified by the test client or in accordance with the specified requirements. Basic test workflow
The relationship between software development process activities and software testing process activities is shown in Figure 1. Requirements analysis
Software development process
Test preparation
Software testing process
Software design
Test plan
Coding implementation
Test design
Figure 1 Relationship diagram between software development process and testing process Revision
Test result judgment and processing
Test analysis and evaluation
Test report/evaluation report
Each activity involved in the software testing process should be described according to the structure of input, task and output. The test workflow includes seven stages: test preparation, test plan formulation, test design, test execution, test result judgment and processing, test analysis and evaluation, and test report preparation.
The basic software testing workflow and its activities are shown in Figure 2. The software testing workflow described in this standard is applicable to software testing of the test types described in Chapter 1, regardless of the test method used.
CB1360-2002
Test preparation
Develop test plan
Test plan review
Review passed?
Test design, establish test environment, determine resources, prepare or generate test cases, determine test procedures, develop test auxiliary programs. Is code review required?
Code review
Review passed?
Test execution
Meet requirements?
Strength test is required
Strength test
Meet requirements?
Test result determination and processing
Test analysis and evaluation
Prepare test report
Defect tracking
Review after modification
Regression test or retest
Modify, supplement test, increase strength
Software testing workflow diagram
4.7 Disposal of tested software
4.7.1 Methods and procedures
CB1360-2002
In order to ensure the security and identifiability of the tested software, as well as the certainty, traceability of the test status and the continuity and consistency of configuration management, the tester shall formulate methods and procedures for the disposal of the tested software. 4.7.2 Receipt of tested software
The requirements for receiving tested software are as follows:
When receiving the tested software, the tester shall check the status of the tested software and related documents and fill in the tested software entry and exit registration form. The format of the entry and exit registration form for the tested software is shown in Appendix E: b)
Record the receiving status of the tested software:
The tested software shall be registered by the sample administrator of the testing party, kept in a dedicated machine/cabinet/media, and clearly marked, with clear configuration status, c)
The configuration identification is clear.
Identification of the tested software
The identification requirements of the tested software are as follows:
The testing party shall clearly and significantly identify the tested software when receiving it: a)
If the tested software is a single piece or multiple pieces and has been identified, it only needs to be identified on the entry and exit registration form of the tested software: b)
The identification of the tested software shall be consistent with the identification of the media to ensure the uniqueness, integrity and traceability of the identification. c)
Test status identification of the tested software
The test status identification requirements of the tested software are as follows: a) To be tested:
b) Tested (passed or failed).
4.7.5 Handover of the tested software
After the software test is completed, the test client shall sign the tested software entry and exit registration form, and then the tested software and the software test report shall be handed over to the test client by the test party. 4.8 Basic requirements for testing
The main purpose of software testing is to determine the correctness, completeness, consistency, compliance, etc. of the software implemented according to the software requirement specifications and design specifications, and try to find problems in the following aspects: Software requirement characteristics and their corresponding descriptions (such as inactivity, activity waiting, activity processing and other signals): a)
Requirement specifications for valid input (including interface requirement specifications) or user manuals, including design documents when necessary b)
And source code, etc.:
Software-related usage and operating procedures: c)
Algorithms, data structures;
Decision boundaries of software control logic.
4.9 Test records and control
4.9.1 Contents of test records
Test records should include the following contents:
Test contract review records and reports;
Test plan, test procedures and test instructions, etc.: Test process data:
Test input/output data;
Expected results;
Test results:
Test problem report sheet;
Analysis and evaluation report;
CB1360-2002
Test report:
Related supplementary agreements, minutes and problem instructions formed during the testing process. 4.9.2 Test record and control requirements
Test record and control requirements are as follows:
a) The tester shall form a complete test record during the test process:b)
The tester shall truthfully record various phenomena and output information during the test process. If the output information is too long or too much, it shall be recorded one by one and explained in detail:c)
During the on-site test process, if the test driver, test tool, test case, database, etc. are changed or the original code is modified, the status before and after the modification shall be compared and recorded to organize retesting or regression testing. 4.9.3 Test record management
Test records shall be archived and preserved in accordance with the control requirements of documents and materials to ensure the reproducibility of the test. Test records can be provided to the test client as part of the test report. 4.10 Test deviation handling
4.10.1When deviations from regulations and procedures or disagreements occur during the test process, the tester shall conduct timely analysis and take effective measures according to the prescribed procedures to prevent the spread and expansion of accidents. 4.10.2 When deviations from regulations and procedures or divergences occur during the test process, the technical person in charge or the test team leader of the test party shall organize relevant personnel to analyze the causes and responsibilities of deviations from regulations and procedures or test divergences, formulate corrective measures, and fill in the report on corrective measures for deviations from regulations and procedures or test divergences. The report format is shown in Appendix F. 4.10.3 After the report on corrective measures for deviations from regulations and procedures or test divergences is approved in accordance with the prescribed procedures, the test team shall implement corrections and handle them.
4.10.4 When the correction of deviations from regulations and procedures or test divergences affects the quality of the test work, it shall be promptly fed back to the test client in written form in accordance with the prescribed procedures.
4.10.5 When necessary, the corrective measures for deviations from regulations and procedures or test divergences shall be reviewed. 5 Detailed requirements
5.1 Test workflow
5.1.1 Test preparation
Before the start of the test work, the test party shall make overall arrangements and make test preparations. During the test preparation process, the test party shall negotiate and reach an agreement with the test client on relevant matters. 5.1.1.1 Input
Written or oral test commission from the test client. 5.1.1.2 Task
5.1.1.2.1 After accepting the test commission, the test party shall sign a test contract with the test client on the basis of consensus on relevant issues.
The test party shall establish a software testing group in accordance with the requirements of 4.2, clarify the composition of the test group, the responsibilities and authority of each member and 5.1.1.2.2
working methods.
5.1.1.3 Output
Test contract or agreement and a series of related additional instructions. Usually, in addition to general terms, a test contract should also 5.1.1.3.1
specify and explain the following contents:
a) Test object and scope;
Test nature and purpose:
c) Test progress requirements and work arrangements;
d) Standards and passing criteria for the test: 81
Specify and explain the following contents:
a) Test object and scope;
Test nature and purpose:
c) Test progress requirements and work arrangements;
d) Test standards and passing criteria: 81
Specify and explain the following contents:
a) Test object and scope;
Test nature and purpose:
c) Test progress requirements and work arrangements;
d) Test standards and passing criteria: 88 Basic requirements for testing
The main purpose of software testing is to determine the correctness, completeness, consistency, compliance, etc. of the software implemented according to the software requirement specifications and design specifications, and to try to find problems in the following aspects: Software requirement characteristics and their corresponding descriptions (such as inactivity, activity waiting, activity processing and other signals): a)
Requirements specification for valid input (including interface requirements specification) or user manual, including design documents when necessary b)
And source code, etc.:
Software-related usage and operating procedures: c)
Algorithms, data structures;
Decision boundaries of software control logic.
4.9 Test records and control
4.9.1 Contents of test records
Test records should include the following contents:
Test contract review records and reports;
Test plan, test procedures and test instructions, etc.: Test process data:
Test input/output data;
Expected results;
Test results:
Test problem report sheet;
Analysis and evaluation report;
CB1360-2002
Test report:
Related supplementary agreements, minutes and problem instructions formed during the testing process. 4.9.2 Test record and control requirements
Test record and control requirements are as follows:
a) The tester shall form a complete test record during the test process:b)
The tester shall truthfully record various phenomena and output information during the test process. If the output information is too long or too much, it shall be recorded one by one and explained in detail:c)
During the on-site test process, if the test driver, test tool, test case, database, etc. are changed or the original code is modified, the status before and after the modification shall be compared and recorded to organize retesting or regression testing. 4.9.3 Test record management
Test records shall be archived and preserved in accordance with the control requirements of documents and materials to ensure the reproducibility of the test. Test records can be provided to the test client as part of the test report. 4.10 Test deviation handling
4.10.1When deviations from regulations and procedures or disagreements occur during the test process, the tester shall conduct timely analysis and take effective measures according to the prescribed procedures to prevent the spread and expansion of accidents. 4.10.2 When deviations from regulations and procedures or divergences occur during the test process, the technical person in charge or the test team leader of the test party shall organize relevant personnel to analyze the causes and responsibilities of deviations from regulations and procedures or test divergences, formulate corrective measures, and fill in the report on corrective measures for deviations from regulations and procedures or test divergences. The report format is shown in Appendix F. 4.10.3 After the report on corrective measures for deviations from regulations and procedures or test divergences is approved in accordance with the prescribed procedures, the test team shall implement corrections and handle them.
4.10.4 When the correction of deviations from regulations and procedures or test divergences affects the quality of the test work, it shall be promptly fed back to the test client in written form in accordance with the prescribed procedures.
4.10.5 When necessary, the corrective measures for deviations from regulations and procedures or test divergences shall be reviewed. 5 Detailed requirements
5.1 Test workflow
5.1.1 Test preparation
Before the start of the test work, the test party shall make overall arrangements and make test preparations. During the test preparation process, the test party shall negotiate and reach an agreement with the test client on relevant matters. 5.1.1.1 Input
Written or oral test commission from the test client. 5.1.1.2 Task
5.1.1.2.1 After accepting the test commission, the test party shall sign a test contract with the test client on the basis of consensus on relevant issues.
The test party shall establish a software testing group in accordance with the requirements of 4.2, clarify the composition of the test group, the responsibilities and authority of each member and 5.1.1.2.2
working methods.
5.1.1.3 Output
Test contract or agreement and a series of related additional instructions. Usually, in addition to general terms, a test contract should also 5.1.1.3.1
specify and explain the following contents:
a) Test object and scope;
Test nature and purpose:
c) Test progress requirements and work arrangements;
d) Standards and passing criteria for the test: 88 Basic requirements for testing
The main purpose of software testing is to determine the correctness, completeness, consistency, compliance, etc. of the software implemented according to the software requirement specifications and design specifications, and to try to find problems in the following aspects: Software requirement characteristics and their corresponding descriptions (such as inactivity, activity waiting, activity processing and other signals): a)
Requirements specification for valid input (including interface requirements specification) or user manual, including design documents when necessary b)
And source code, etc.:
Software-related usage and operating procedures: c)
Algorithms, data structures;
Decision boundaries of software control logic.
4.9 Test records and control
4.9.1 Contents of test records
Test records should include the following contents:
Test contract review records and reports;
Test plan, test procedures and test instructions, etc.: Test process data:
Test input/output data;
Expected results;
Test results:
Test problem report sheet;
Analysis and evaluation report;
CB1360-2002
Test report:
Related supplementary agreements, minutes and problem instructions formed during the testing process. 4.9.2 Test record and control requirements
Test record and control requirements are as follows:
a) The tester shall form a complete test record during the test process:b)
The tester shall truthfully record various phenomena and output information during the test process. If the output information is too long or too much, it shall be recorded one by one and explained in detail:c)
During the on-site test process, if the test driver, test tool, test case, database, etc. are changed or the original code is modified, the status before and after the modification shall be compared and recorded to organize retesting or regression testing. 4.9.3 Test record management
Test records shall be archived and preserved in accordance with the control requirements of documents and materials to ensure the reproducibility of the test. Test records can be provided to the test client as part of the test report. 4.10 Test deviation handling
4.10.1When deviations from regulations and procedures or disagreements occur during the test process, the tester shall conduct timely analysis and take effective measures according to the prescribed procedures to prevent the spread and expansion of accidents. 4.10.2 When deviations from regulations and procedures or divergences occur during the test process, the technical person in charge or the test team leader of the test party shall organize relevant personnel to analyze the causes and responsibilities of deviations from regulations and procedures or test divergences, formulate corrective measures, and fill in the report on corrective measures for deviations from regulations and procedures or test divergences. The report format is shown in Appendix F. 4.10.3 After the report on corrective measures for deviations from regulations and procedures or test divergences is approved in accordance with the prescribed procedures, the test team shall implement corrections and handle them.
4.10.4 When the correction of deviations from regulations and procedures or test divergences affects the quality of the test work, it shall be promptly fed back to the test client in written form in accordance with the prescribed procedures.
4.10.5 When necessary, the corrective measures for deviations from regulations and procedures or test divergences shall be reviewed. 5 Detailed requirements
5.1 Test workflow
5.1.1 Test preparation
Before the start of the test work, the test party shall make overall arrangements and make test preparations. During the test preparation process, the test party shall negotiate and reach an agreement with the test client on relevant matters. 5.1.1.1 Input
Written or oral test commission from the test client. 5.1.1.2 Task
5.1.1.2.1 After accepting the test commission, the test party shall sign a test contract with the test client on the basis of consensus on relevant issues.
The test party shall establish a software testing group in accordance with the requirements of 4.2, clarify the composition of the test group, the responsibilities and authority of each member and 5.1.1.2.2
working methods.
5.1.1.3 Output
Test contract or agreement and a series of related additional instructions. Usually, in addition to general terms, a test contract should also 5.1.1.3.1
specify and explain the following contents:
a) Test object and scope;
Test nature and purpose:
c) Test progress requirements and work arrangements;
d) Standards and passing criteria for the test: 8
Tip: This standard content only shows part of the intercepted content of the complete standard. If you need the complete standard, please go to the top to download the complete standard document for free.