Evaluation method of residual intelligibility for analog-voice scramblers (RIAVS)
Some standard content:
Military Standard of the Electronic Industry of the People's Republic of China FL5821
SJ20041—92
Evaluation method of residual intelligibility for analog-voice scramblers (RIAvS)Published on February 1, 1992
China Electronics Industry Corporation
Implemented on May 1, 1992
Military Standard of the Electronic Industry of the People's Republic of China Evaluation method of residual intelligibility for analog-voice scramblers (RIAVS)1 Subject content and scope of application
1.1 Subject content
SJ20041—92
This standard specifies the test information materials, test conditions, implementation procedures and calibration scoring methods for the subjective evaluation method of residual intelligibility of voice confidentiality signals of analog confidentiality machines, and makes recommendations on whether the confidentiality level is acceptable to users based on the residual intelligibility score.
1.2 Scope of application
This standard applies to the subjective evaluation of residual intelligibility of voice confidentiality signals of voice analog scrambling systems and the indirect evaluation of confidentiality.
This standard does not apply to the sound quality evaluation of voice analog scrambling systems. 2 Reference standards
GB2019
GB3785
GB7341
GB7614
GB/T13504
3 Terminology
3.1 Plain text information
Basic parameters and technical requirements of tape recordersElectroacoustic performance and test methods of sound level meters
Audiometer
Calibrating wide-band artificial ears for audiometric headphonesChinese language intelligibility diagnosis rhyme test method
Refers to the original language information that has not been scrambled by the analog voice scrambling system. 3.2 Information unit
The language information unit used in the test that can be pronounced and recorded independently is called an information unit or an information element. In this standard, it specifically refers to two-syllable words, four-syllable phrases or idioms and four-digit number strings. 3.3 Plaintext information set
refers to the set of plaintext information of a certain scale composed of the above-mentioned specific language information units. In this standard, it refers to the set of two-syllable words, four-syllable phrases or idioms issued by China Electronics Industry Corporation on February 1, 1992 and implemented on May 1, 1992
SJ20041-92
and four-digit number strings composed of ten numbers (0, 1, .., 9). 3.4 Ciphertext information
refers to the language information that is completely incomprehensible after the plaintext information is scrambled by the analog telephone machine. 3.5 Ciphertext information set
The set of all ciphertext information units corresponding to a given plaintext information set. 3.6 Remaining intelligible information set
If the number of units in the plaintext information set is C and the number of units in the ciphertext information set is S, the difference between the two, C-S, is called the remaining intelligible information set, denoted by R.
3.7 Residual intelligibility
The percentage of the information set R with residual intelligibility in the total plaintext information set C is called percentage residual intelligibility, or residual intelligibility for short, and is denoted by RI.
3.8 Subset
The subgroup into which the plaintext information set is divided according to certain principles. 3.9 Test table
A test table formed by randomly changing the order of information units in a given plaintext information set, which can be used for individual tests and score statistics.
3.10 Public test table
A set of information units that can be directly selected from it by the listener, which he thinks is being sent. In this standard, it is the set of basic information materials. 4 Structure of information material set
4.1 Selection of information materials
Basic information materials are selected from two-syllable words with a high occurrence rate and easy to hear when pronounced in isolation, as well as four-syllable phrases or idioms. It also includes a number string formed by ten numbers. They constitute three small-scale basic information material sets. 4.2 Disyllabic words
Select 100 disyllabic words. Two basic sets of disyllabic words are formed. Each set contains 50 words, half of which are military terms. The other half are general terms. Every five words with a large pronunciation distance form a subset, and the whole set has ten subsets. The word order and (or) subset order are randomly changed to form a test table, see Chapter A1 of Appendix A (Supplement). 4.3 Four-syllable phrases and idioms
have the same structure as the disyllabic word set. There are also two basic sets, half of which are military phrases and the other half are common idioms, see Chapter A2 of Appendix A (Supplement). 4.4 Four-digit number strings
consist of ten groups of four-digit numbers, each group has 10 four-digit number strings. The appearance of the ten numbers in each group is balanced, that is: 0.1.2.3..9 appears four times each. In order to take into account the appropriateness of the test time, it is divided into two groups, each of which contains five subsets. In addition, considering the equal chance of the influence of the previous and next pronunciations, the adjacent chances between the previous and next two numbers, including the ten numbers themselves, are balanced, that is, each adjacent chance appears three times, see Chapter A3 of Appendix A (Supplement). 4.5 Training Tables
For RIAVS testing, it is particularly important for testers to get used to the confidential voice of the tested system. Therefore, special training tables are used. They have the structural characteristics of repeated pronunciation. a. The number table takes five forms: XXXX, XXXY, XXYY, XYYY and XYXY (YXYX). Ten are also one 2
SJ20041—92
subset, a total of 5 subsets. The appearance of the 10 numbers is basically balanced, see Chapter B3 of Appendix B (Supplement). b. The idiom table is in the following five forms: XXYY (e.g., colorful), XXYY (e.g., flourishing), XYZZ (e.g., fanciful), XYXZ (e.g., reducing sincerity), and XYZY (e.g., getting by). Ten idioms of each form are selected to form a subset, for a total of five subsets, which have the same structure as the test set, see Chapter B2 of Appendix B (Supplement). c. There are two disyllabic word tables. The structure is the same as the test set, see Chapter B1 of Appendix B (Supplement). Information units that have appeared in the training table, whether they are digit strings or words, will no longer appear in the test information set.
5 Test method
This standard only gives the general conditions and procedures for the residual intelligibility test method. 5.1 Test conditions
5.1.1 Pronunciation environment
If the analog voice confidentiality machine is directly sent to the machine with a microphone in the laboratory, the average sound pressure level of the background noise should be lower than 40dB (A weighted, slow gear); the reverberation time should be lower than 0.5s (500Hz). The sound level meter used should meet the requirements of GB3785.
If a recorded tape is used to send sound, the average signal level sent by the recorder should ensure that the signal-to-noise ratio is better than 45dB. The voice tape is best recorded in an anechoic room or in an environment where the reverberation time can be ignored. In addition, the recording equipment including the telephone should meet the requirements of GB2019.
5.1.2 Speaker
At any time, when conducting residual intelligibility tests on any analog voice confidentiality equipment, at least one middle-aged or young male and one female voice should be used as the speaker. Standard Mandarin should be spoken. There should be no abnormal timbre characteristics. Before formal pronunciation, sufficient training should be ensured to be able to pronounce according to the requirements of Article 5.3.
5.1.3 Test environment
Generally, dual headphones with earmuffs should be used for listening. The unevenness of the frequency response within 200-5000Hz should not exceed 3dB, and the environmental noise should be lower than 50dB. When using speakers for listening under unavoidable circumstances, the background noise of the sound field should be lower than 40cB (A sound level), the room reverberation time should not exceed 0.5s (500Hz), and the unevenness of the sound field distribution should not exceed 3dB. Regardless of the way of listening, visual interference and any sudden noise, especially various impact sounds, should be avoided. For systems equipped with headphones and other listening equipment, the system's own listening equipment should be used for audio testing. 5.1.4 Listeners
The listener should have a junior high school education or above and be able to correctly listen to and distinguish Mandarin. When they are tested with a pure tone audiometer that complies with GB7341, the average hearing loss at frequencies of 250, 500, 1000, 2000, and 40000 Hz does not exceed 1 odB.
The test team should consist of at least 10 young and middle-aged male and female members. 5.2 Test Tables
Test pronunciation tables are generally formed by randomly selecting test information units from each subset of the plain text information set in turn, with one unit from each subset in each round, and at least five rounds for each table. After the pronunciation table is formed, a corresponding formatted recording table should be prepared so that the listener can answer and record in the prescribed format. For any test, the test of number strings and words should be included, and at least the following should be used: a. Two number string word tables for male and female voices.
b. Two two-syllable word tables for male and female voices and two four-syllable phrase or idiom word tables. 3
5.3 Pronunciation of test word tables
5.3.1 Pronunciation method
SJ20041—92
Before pronunciation, the speaker should first report the pronunciation table number in plain language. Any test information unit is pronounced three times in succession, and at the beginning of each test information unit's three pronunciations, the pronunciation of the plaintext number "the Xth" ("X" is a number) should be inserted. 5.3.2 Pronunciation speed
The pronunciation and recording time distribution of number strings and four-syllable phrases or idioms are shown in Table 1. Table 1 Pronunciation and recording time of number strings and four-syllable words Sequence number
First pronunciation
Second pronunciation
The pronunciation and recording time distribution of disyllabic words are shown in Table 2. Pause
Third pronunciation
Table 2 Pronunciation and recording time of disyllabic words
Sequence number
5.3.3 Pronunciation of numbers
First pronunciation
Second pronunciation
Third pronunciation
About 11s
Pronounce according to the commonly used calling method for communication. That is, "0\ is pronounced as "洞", "1" is pronounced as "么", "2" is pronounced as "两", "7" is pronounced as "拐", and "9" is pronounced as "勾". The rest are pronounced as usual. 5.3.4, Pronunciation sound level
5.3.4.1 When using the phone to input directly, the speaker should use natural strength to pronounce. When the phone is 4cm away from the lips, the average sound pressure level of the three consecutive calls of the test information unit with a large pronunciation sound level is measured to be about 94dB (A weighting, slow gear). When the pronunciation with such strength does not meet the volume requirements of the device, the volume controller of the device itself can only be adjusted, and the volume requirements cannot be met by artificially increasing or decreasing the pronunciation force. Special emphasis is placed on natural and smooth pronunciation, and the volume cannot be high or low, and the speed cannot be fast or slow. Avoid re-reading and intonation. 5.2 .4.2 When using a recorded tape as input, the volume of the recorder should be adjusted to meet the requirements. 5.4 Audiometry Recording
5.4.1 Listening Sound Level
When listening with headphones, the sound volume is generally measured using a simulated ear. The sound pressure level sent by the headphone diaphragm should be around 75dB. When using a loudspeaker to send sound, the sound pressure level at the listener's position should be around 65dB. Before the test, the test host should use the same test information unit as in 5.3.41 to adjust the volume in advance. The listener is not allowed to adjust the volume during the test. 5.4.2 Recording Method
5.4.2.1 For two-syllable words and four-syllable phrases or idioms, a clearly defined public test list is used for selection and answering, that is, the listener judges the stimulus information heard from a subset of the public test list. Unit, and then write down the serial number of the heard stimulus information unit item in the corresponding subset on the answer record sheet (numbers of 1, 2, 5). When you are not sure, you should also mark the closest guessed information unit serial number.
5.4.2.2 For the four-digit string, the four digits must be recorded completely. When it is difficult to judge, the closest guess string should also be recorded, and no blanks should be left.
The record should not be easily changed, nor should it refer to the records of others. 5.4.3 Preparation work
5.4.3.1 Training of test team members
SJ20041-92
For any residual intelligibility test of the scrambling device, sufficient and repeated training must be carried out using the training table given in Appendix D (reference). This training is required to achieve: a. Clarify the purpose of the test and enhance the sense of responsibility, b. Get used to the scrambled voice of the device under test, c. Master the essentials, methods and requirements for recording answers. Training is best conducted in a clear and secret contrast mode, and should be repeated many times, first clear then secret and first secret then clear. For each information set material and each training method, it is recommended to use two training tables for male and female voices. 5.4.3.2 Prepare test tables First, determine the number of male and female test tables to be used in a test, and then form the test pronunciation table based on the three basic information set materials in Appendix A (Supplementary). For word information materials, form them according to the method of Article 5.2. And assign table numbers to each table, see Appendix C (Reference). For digital string information materials, it is required to randomly change the subset number and the order of information units in each subset at the same time. Before the test, a formatted recording table for each information material should be prepared for each listener [see Appendix D (Reference) and handed to each listener.
The pronunciation table of two-syllable words, four-syllable phrases or idioms, the pronunciation table of number strings, and the basic information collection material table of digital applications in Chapter A3 of Appendix A (Supplement) shall not be exposed to the listener before the entire test listening is completed. 5.4.4 Verification of the record sheet
When verifying the record sheet, one person should repeat the pronunciation of the test pronunciation table or replay the original plain text voice under the direct participation of the test host, and the team members should verify each other. They shall not verify their own records. It is sufficient to mark the correct answers, and it is not necessary to mark the wrong answers. When verifying the digital application, a short line can be drawn under the single digit position of the correct answer, such as: "2864", 2, 6, and 4 are the correct answer numbers. 6 Scoring
When the score is counted, the set of all stimulus information units in a pronunciation table is regarded as the set of plain text information. The set of all wrong answer units recorded is regarded as the set of cipher text information. In general, the scores of male and female pronunciation tables are counted separately, and then the total average score is given. 6.1 Word score
When using the public test table for selective answering. The contribution of the visual channel exceeds 60%. Therefore, the score calculated by the definition RI-CS
·（1）
is deducted from the probability of guessing the answer, and then corrected by the correction factor (0.35/n)\, where n is the scrambling dimension. Finally, the scores of the residual intelligibility RI of disyllabic words and the residual intelligibility RI of phrases or idioms are given by the following formula: RIa=(0.35CS
P)×100
(or RIu）
Where: C=50. When testing words, the number of selective answers of the subset information unit is 5. Therefore, P in formula (2) is =0.2. When the product of the time period scrambling number N and the frequency band scrambling number f of the security machine is Nf≥6, formula (2) is valid. 5
The scoring steps are:
SJ20041--92
a. Confirm the simulated scrambling dimension n of the system under test; the complexity factor Nf. When Nf≥6, calculate the residual intelligibility RIa of two-syllable words and the residual intelligibility RIid of four-syllable phrases or idioms of male and female voices respectively according to the following steps. b. Calculate the average residual intelligibility score of all valid listeners M for the k-th pronunciation table according to formula (2): RIai.k
(or Rlak)
i=1,2,.....M
Wherein, Ci, Si are the number of information units of the actual valid plaintext information set and ciphertext information set of the i-th test table respectively. c. Calculate the average residual intelligibility score of all pronunciation tables. RIa
or RId
（5）
i-1,2,......K
where: K is the number of test tables that are actually valid. d. According to formula (3), calculate
RIa= (0-35) (RI - P,) × 100(0.35)(RI-P.)×100
e. Calculate the average residual intelligibility score RIa or RId of male and female voices. 6.2 Scores of residual intelligibility of digit strings
The following correction formula is used to calculate the scores of residual intelligibility RI of digit strings for male and female voices respectively: RIai=
W(R,- P.) ×100
i=1,2.3,4
（6）
Wherein, R, is the correct answer rate of the i-digit digit, and W, is the probability of guessing the remaining digits under the condition that the correct answer of the digit has occurred. When the digit is correctly answered, W,=0.001, and when two digits are correctly answered, W,=0.01. By analogy, W,=0.1,W,1.
P, is the contribution of the probability of guessing correctly to the correct answer. Among the four-digit numbers, the contribution of one correct guess is P, = 0.1, the contribution of two correct guesses is Pz = 0.01, and so on, Pr = 0.001, Pr = 0.0001. SJ20041--92
When the correct answer rates of one, two, three and four digits R,, R, R, and R are obtained respectively, the following formula can be used: RI=(0.001R,+0.01R,+0.1R,+R,0.0004)×100 to obtain the final score of the digital application.
The scoring steps are as follows:
a. Calculate the correct answer rates of various digits of the mth listener on the kth record sheet. Correct answer rate of one digit
Where: C=100.
Correct answer rate for two digits
CQ1+Q23+Q2+Q3t)
（10)
Where: Q1(i-2, 3, 4), Q2, Q represent various two-digit correct answer numbers respectively, for example, Q1 represents the first and second correct answer numbers, Q1 represents the first and third correct answer numbers, and so on. Correct answer rate for three digits
Correct answer rate for four digits
(Qus+Q+Qiu+Q3
b. Calculate the average correct answer rate for each digit of all listeners M on the k-th record sheet. Rk
Where: for 1=1, 2, 3, 4, calculate respectively, c. Calculate the average correct answer rate for each digit of all record sheets of the same test system. K
Where, i=1, 2, 3, 4.
· (14)
（15)
d. Calculate RIa according to formula (8) or (9).
e: Calculate the average score RIa of male and female voices. 6.3 Comprehensive score of residual intelligibility
SJ20041—92
When the score contribution of two-syllable words, four-syllable phrases or idioms and number strings is combined, the following formula RI=0.5RI+0.5RI+RI)
is used to give the total residual intelligibility score.
6.4 Handling of negative scores
During the statistical process, the score is calculated as a real number. If the final result is a negative number, this test is invalid. Residual intelligibility level
·(16)
Based on the evaluation results of residual intelligibility, this standard stipulates the following levels of whether the residual intelligibility of the simulated security machine is acceptable to users.
RI≥25%
25%>RI≥15%
15%>RI≥5%
5%>RI≥1%
RIAVS test set of two-syllable words
Basic word set 1
A1.2 Basic word set 2
SJ20041—92
Appendix A
Basic letter material set
(Supplement)
RIAVS test set of four-syllable phrases and idioms A2.1
Idiom and phrase set 1
(1) 1. Military secrets
(2) 1. Long-range missile
(3) 1. Overjoyed
（4）1. Be vigilant
（5）1. Track and pursue
（6）1. Insignificant
（7）1. Aircraft carrier
（8）1. Young and strong
2. Tit for tat||2. Let a hundred flowers bloom
2. Reconnaissance satellite
2. Self-sacrifice
2. Preserve strength
2. Add unnecessary details
2. Launch an attack
2. Change one's mind
3. Border defense force
3. Air defense system
3. Overcome all difficulties
3. Knowingly commit a crime
3. Rest easy
3. Quick battle and quick victory 3. Uniquely endowed
3. Aerial shooting
4. Aboveboard5. Make a feint to the east and attack in the west
4. Share weal and woe
4. Coordinate operations
4. Lure the enemy deep into our territory
4. Strive to be the first and the last
4. Overcome all difficulties
4. Earth-shaking
4. Lead by example
5. Clear at a glance
5. Turn a blind eye
5. Defeat them one by one
5. Focus on cover
5. Put the wrong hat on someone else
5. Military dock
5. Naval base
(9)1. Punish the past and the future
2. Military-civilian integration||tt ||SJ20041-92
3. compare with
(10)1. a splendid array
2. flawless
3. take the enemy by surprise
A2.2 Idioms and Phrases Collection II
(1)1. do everything possible
(2)1. a constant stream
(3)1. break down into equal parts
(4)1. return home immediately
(5)1. share life and death
(6)1. create something out of nothing
(7)1. consider carefully
(8)1. atomic weapons
(9)1. artillery support
(10)1. core fortifications
2. defense 2. Strategic transfer
2. Easy
2. Great achievements
2. As it should be
2. Discover the target
2. Launch an attack
2. Selfless
2. Streamline the military
2. Long March
3. Prepare for battle
3. Code telegram
3. Hold the ground
3. It is natural
3. Unknown high ground
3. It is difficult to move
3. Sit in a well and look at the sky
3. Rescue on the front line
3. Conspiracy
3. Both ability and political integrity
A3.RI A collection of four-syllable phrases and idioms for AVS test A3.1
First group
A3.2Second group
4. When the enemy advances, we retreat
4. Prepare for battlebZxz.net
4. In groups
4. Deploy for battle
4. Some are lost
4. Go back and forth
4. Strengthen patrols
4. Replenish ammunition
4. Stay on standby
5. Support the government and love the people
5. Be thrifty
5. Seek truth from facts
5. Not afraid of habitat
5. Stop advancing
5. Be willing
5. Anti-aircraft artillery
5. looking down from a high place
4. jubilant
5. field exercise
4. shortwave communication
4. depth charge
5. water reaches the beam
5. countless
B1 double-single syllable word set
B1.1 set one
(1) 1. safe 2. keep
camp 2. regret
(2) 1. no matter
(3) 1. promise
(4) 1. nearby
(5) 1. drama: 2. meeting
(6) 1. nervous
safe 2. problem
(7) 1. mother||tt ||(8) 1. Climate
(9) 1. Provincial Party Committee
(10) 1. Stable 2. Factory
SJ20041-92
Appendix B
Training Table
(Supplement)
4. Competition 5. Expression
4. Success 5. Processing
4. Law 5. Aspect
4. Actual 5. As if
4. Inspection, 5. Traffic
4. Remember 5. Democracy
4. Calm 5. Dispatch
Electricity 5. Great
4. As long as 5. Freedom
Note: The pronunciation table for training can be formed by this table using a random sampling method. The record can be answered directly on the table using the "marking method". B1.2 Collection 2
（1）1. Comfort
（2）1. Supplement
（3）1. Reach
（4）1. Place
（5）1. Garden
（6）1. Spirit
（7）1. Goal
（8）1. Strange
（9）1. Life
（10)1. Interlude
B2 Idiom Collection
: 5. origin
4. meet 5. education
4. teacher 5. order
4. comment 5. platoon leader
4. endure 5. mountain area
4. assault 5. danger
(1) 1. vigorous
2. various
3. mighty 4. original 5.densely packed (2) 1. upright
2. full
3. colorful
4. secretive 5. busy
(3) 1. thriving
3. closely related
2. reluctant to leave
4. endless torrent 5. full of life
(4) 1. enjoy
2. heart to heart 3. take everything into consideration
14. be complacent 5. never forget
(5) 1. a thousand miles away
2. daydream 3. Spirited 4. Full of life 5. Eyeing with eager eyes (6) 1. Dusty 2. Well-mannered 3. Cautious 4. Elated 5. Eyed with eager eyes (7) 1. Not a good person 2. Selfish 3. Perfect 4. Sincere 5. Vivid (8) 1. Hit the target every time (9) 1. Get by 2. Be of one mind 3. Be epic 4. Disappear without a trace 5. Be diligent and frugal 2. Seek truth from facts 3. Follow the crowd 4. Go back on one's words 5. Use a trick against someone
Tip: This standard content only shows part of the intercepted content of the complete standard. If you need the complete standard, please go to the top to download the complete standard document for free.