Definition of archived data format for space-based high-energy astronomy
Some standard content:
ICS07.040
National Standard of the People's Republic of China
GB/T37846—2019
Definition of archived data format for space-based high-energy astronomy2019-08-30Release
State Administration for Market Regulation
Standardization Administration of the People's Republic of China
2019-12-01Implementation
GB/T37846—2019
Normative Reference Documents
Terms and Definitions
4 Overall Structure
Basic Unit
5.2 Basic File Header
5.3 Basic Data Array
Free Group
6 Normative Extension Unit||tt| |Structure and classification
6.2 Standard extension unit
Special record
Keyword record
Record method
Keyword name
Value indicator
8.4 Keyword value and annotation
8.5 Unit
Appendix A (Normative Appendix) Keywords
Appendix B (Informative Appendix) Registered normative extension types Appendix C (Informative Appendix)
References
This standard was drafted in accordance with the rules given in GB/T1.12009. This standard was proposed by the Chinese Academy of Sciences.
This standard is under the jurisdiction of the National Technical Committee for Space Science and Its Application Standardization (SAC/TC312). GB/T37846—2019
This standard was drafted by the Institute of High Energy Physics, Chinese Academy of Sciences, the National Space Science Center, Chinese Academy of Sciences, and the Space Application Engineering and Technology Center, Chinese Academy of Sciences.
The main drafters of this standard are: Song Liming, Zheng Shijie, Liu Yuan, Ge Mingyu, Li Chengkui, Nie Jianyin, Xu He, Tong Jizhou, Ji Zhen, Li Yunlong, Hei Baoqin.
1 Scope
Archive format of space high-energy astronomical data
GB/T37846—2019
This standard specifies the general format of archiving space high-energy astronomical data, including overall structure, basic unit, standard extension unit, special record and keyword record, etc.
This standard applies to the production, release, use and archiving of space high-energy astronomical data, and other astronomical observation data can be implemented as a reference. Note: The high-energy astronomical data in this standard mainly include observation data generated by astronomical research in the X-ray and gamma-ray bands. 2 Normative reference documents
The following documents are indispensable for the application of this document. For all referenced documents with dates, only the versions with the dates are applicable to this document. For any undated referenced document, its latest version (including all amendments) applies to this document. GB/T 30114.1 Space science and its application terminology Part 1: Basic general GB/T 30114.3 Space science and its application terminology Part 3: Space astronomy IEEE 754-2oo8 Information Technology Microprocessor Systems-Floating-Point Arithmetic 3 Terms and definitions
The terms and definitions defined in GB/T 30114.1 and GB/T 30114.3 and the following terms and definitions apply to this document 3.1
ASCII text ASCII text
Text composed of characters whose position in the ASCII (American Standard Code for Information Interchange) table is between decimal 32126 or hexadecimal 20~7E. 3.2
keyword record
Keyword record
Text consisting of 80 bytes of ASCII characters, used to describe the keyword name, value and comments. 3.3
FITS blockFITSblock
A record block consisting of 2880 consecutive bytes (2880X8 bits) in a FITS (Flexible Image Transport System) file. bzxZ.net
Headerblock
A FITS block consisting of 36 keyword records. 3.5
Header
Description consisting of one or more header blocks, describing the subsequent data structure and data content. 3.6
Datablockdatablock
FITS block containing record data.
GB/T37846—2019
HDU header and data unit
The structure consists of the file header and the corresponding data unit, which can exist at the same time or only consist of the file header. 3.8
indexed keyword
Indexed keyword
Keyword type consisting of the same root and a positive integer added in ascending order. 3.9
Heap area
Data supplement area after the basic data table. 4 Overall structure
The overall structure of the data archive file is:
a) Basic unit:
b) Standard extension unit (optional);
c) Special record (optional).
The data archive file should start with a basic unit, followed by multiple standard extension units, and can be ended by a special record. The content of the data archive file is recorded by an integer number of FITS blocks, and the number of FITS blocks is not mandatory. The overall structure is shown in Table 1. Table 1 Overall structure of archive file
Basic unit
Normative extension unit 1
Normative extension unit n
Special record
5 Basic unit
5.1 Structure
Basic file header
Unit header 1
Unit header n
Basic data array or free group
Data sequence 1
Data sequence n
Integer number of FITS blocks
See Chapter 5 for details
See Chapter 6 for details
See Chapter 7 for details
Basic unit The element should start with a basic file header, which can be followed by a basic data array or a free group, but the basic data array and the free group cannot exist at the same time, and different keyword records should be used in the basic file header to describe the following content. 5.2 Basic file header
5.2.1 Basic data array When the basic file header corresponding to the basic data array is followed by a basic data array, the basic file header should use "\SIMPLE" as the first keyword record (the value should be "T"), and give all mandatory keyword records in the order of Table 2. Other keyword records should be inserted after the last "NAXISn\ keyword record. The unfilled FITS block area after the "END" keyword record should be filled with ASCII spaces. For detailed usage of keywords, see Appendix A.
Last line
Table 2 Mandatory keyword records in the basic file header corresponding to the basic data arrayKeyword name
SIMPLE
BITPIX
NAXISn, n=1,, k (k is the value of the keyword NAXIS) (Other keyword records)
Value indicator
GB/T37846—2019
Keyword value
Values are shown in Table A.1
Non-negative integer not greater than 999
Non-negative integer
This keyword has no value
Note: "NAXIS\" in the third line is the keyword name and the root of the index keyword in the fourth line. The value of this keyword indicates the dimension of the basic data array. If its value is zero, the basic data array should not be included. The usage of \NAXIS\ appearing elsewhere in this standard is consistent with this one. 5.2.2 When the basic file header corresponding to the free group is followed by a free group, SIMPLE\ shall be the first keyword record in the basic file header (the value shall be "T"), and all mandatory keyword records shall be given in the order of Table 3. Other keyword records shall be inserted after the last NAXISn keyword record. The value of the keyword "NAXIS1 shall be O, and other keyword records shall contain "GROUPS" (the value shall be "T"), "PCOUNT" and "GCOUNT". The unfilled FITS block area after the "END\ keyword record shall be filled with ASCII spaces. For detailed usage of keywords, see Appendix A.
Mandatory keyword records in the basic file header corresponding to the free group Table 3
Last row
Keyword name
SIMPLE
BITPIX
NAXISI
NAXISn, n=2, kk is the value of the keyword NAXIS) (other keyword records, which should include GROUPS
PCOUNT
GCOUNT
5.3 Basic data array
Value indicator
Keyword value||tt ||Values are shown in Table A.1
Non-negative integer not greater than 999
Non-negative integer
No value for this keyword
The basic data array consists of a series of array elements, recorded in a continuous data stream, with the first bit of the first data block as the start bit of the stream. The length of each array element should have the same number of bits, declared by the keyword "BITPIX\. The array elements should be stored continuously in positive byte order (high byte first, low byte last). If the last FITS block is not filled, all remaining bits should be filled with binary 0. Multidimensional The storage order of the array is: first store the data of the first dimension, then store the data of the remaining dimensions in sequence. There should be no interval or other special characters between the end of one dimension and the beginning of the next dimension. 3
GB/T37846—2019
5.4 Free Group
A free group can be composed of multiple "groups", each of which consists of a data array and a corresponding set of parameter values. The number of groups in a free group is declared by the keyword "GCOUNT" in the corresponding basic file header, and the number of parameters in each group is declared by the keyword "PCOUNT".||tt| |The parameters and data in a group should be stored consecutively. Starting with the first parameter of the first group, all parameters and data of the group are stored in sequence, followed by the parameters and data of the next group, without any interval or other special characters in between. If the last FITS block is not filled, all remaining bits should be filled with binary 0. 6 Normative extension unit
Structure and classification
The normative extension unit is an extension unit that meets the following requirements: a) It consists of a unit header and a data sequence;
The extension unit type is declared by the keyword "XTENSION" in the unit header. The type name of the normative extension unit is shown in Appendix b)
c) The total length of the data of the extension unit should be declared in the corresponding unit header, see A.1. The normative extension unit that meets the requirements of 6.2 is called a standard extension unit, including: image extension unit, ASCII table extension unit and binary table extension unit.
The normative extension unit should be located after the basic unit. 6.2
Standard extension unit
6.2.1 Image extension unit
6.2.1.1 Unit header
The unit header should use "XTENSION" as the first keyword record (the value should be "\IMAGE", where IMAGE is followed by 3 spaces to fill up 8 bytes), and all mandatory keyword records should be given in the order of Table 4. Other keyword records should be inserted after the "GCOUNT" keyword record. The unfilled FITS block area after the "END" keyword record should be filled with ASCI spaces.
Note: In order to facilitate reading in this standard, the space symbol is used in some contents to represent the space symbol. For detailed usage of the space symbol instead of the "口" keyword in actual operation by the user, see Appendix A.
Table 4 Mandatory keyword records in image extension unit #
Last row
Keyword name
XTENSION
BITPIX
NAXISn, n=1,, k (k is the value of keyword NAXIS) PCOUNT
GCOUNT
(other keywords)
Value indicator
Keyword value
"IMAGE
Value see Table A.1
Non-negative integer not greater than 999
Non-negative integer
No value for this keyword
Data sequence| |tt||The data sequence is located after the unit header. For format and usage, see 5.3, 6.2.2 ASCII table extension unit
6.2.2.1 Unit header
GB/T37846—2019
The unit header should take "XTENSION" as the first keyword record (the value should be "TABLE", where TABLE has 3 spaces after it to fill 8 bytes), and all mandatory keyword records are given in the order of Table 5. The value of the keyword "BITPIX" should be 8. The value of the keyword "NAXIS" should be 2, the value of the keyword "PCOUNT" should be 0, and the value of the keyword "GCOUNT" should be 1. Other keyword records should be inserted after the "TFIELDS\ keyword record. The unfilled FITS block area after the "END\ keyword record should be filled with ASCI spaces.
If the value of the keyword "TFIELDS\" is not zero, the keyword "TBCOLn" (n=1, 2, ..., k, where k is the value of the keyword TFIELDS) and the keyword *TFORMn (n=1, 2.*.k, where k is the value of the keyword TFIELDS) should also be included, and the keyword "TTYPEn" (n=1, 2,, k, where k is the value of the keyword TFIELDS) should be included. Detailed usage of keywords is shown in Appendix A.
Table 5 Mandatory keyword record # in ASCII table extension unit
Last row
Keyword name
XTENSION
BITPIX
NAXISI
NAXIS2
PCOUNT
GCOUNT
TFIELDS
Other keywords, if keyword The value of "TFIELDS" is not zero and should also include TBCOLn, n=1,2,,k, where k is the value of the keyword TFIELDS TFORMn.n=1,2.…,k, where k is the value of the keyword TFIELDS should include:
TTYPEn, n=1,2,***,k, where k is the value of the keyword TFIELDS)END
data sequence
value indicator
keyword value
'TABLE
non-negative integer
non-negative integer
non-negative integer not greater than 999
a string
a string
no value for this keyword
the ASCII table should consist of a two-dimensional array of ASCII characters, with the bytes and number of rows of each row declared by the keywords "NAXIS1" and "NAXIS2\" in the unit header respectively. Each row in the array should have the same number of characters. The first character of the first row should follow the unit header, and the first character of the next row should follow the end of the previous row. If the last FITS block is not filled, all remaining spaces should be filled with ASCII spaces. Each row in the two-dimensional ASCII character array consists of a series of fields. The number, format, starting position and name of the fields in each row are declared by the keywords "TFIELDS", "TFORMn", "TBCOLn" and "TTYPEn" in the unit header respectively. The position of the fields in each row should be consistent with the format of 5
GB/T37846—2019
, should not overlap, and can be discontinuous. ASCII text can be filled before the first field, between two fields or after the last field. A carriage return or line feed character may appear after the last field in each row. The format definition of the field is shown in Table A.2, including:
character domain (Aw). A string of width W; a)
integer domain (Iw). A signed decimal integer of width w, consisting of "ten" (optional) or "one" and the following digits. Space characters can be filled in front of and after the integer. The value of a domain consisting entirely of spaces is O; real number domain (Fw.d, Ew.d, Dw.d). A real number of width w, consisting of a decimal number and an exponent part (optional), right-aligned, and there should be no space characters at the end. The decimal part should contain an explicit decimal point "", starting from the first non-space character, and ending at the decimal point or any character other than the digits "0" to "9" or at the end of the domain. The exponent part can have two formats:
1) a decimal number followed by the symbol "ten" or "one", followed by the exponent expressed as a decimal integer; 2) a decimal number followed by the symbol "E\ or "D", followed by the exponent expressed as a decimal integer (optionally with the symbol "ten" or "two"). The exponent part, if present, terminates at the end of the right-aligned string of digits. 6.2.3 Binary table extension unit
6.2.3.1 Unit header
The unit header should have \XTENSION as the first keyword record (the value should be \BINTABLE), and all mandatory keyword records should be given in the order of Table 6. The value of the keyword BITPIX should be 8, the value of the key NAXIS should be 2, and the value of the keyword GCOUNT should be 1. Other keyword records should be inserted after the "TFIELDS\ keyword record. The unfilled FITS block area after the "END\ keyword record should be filled with ASCII spaces. If the value of the keyword "TFIELDS" is not zero, the keyword \TFORMn\ (n=1, 2,, k, where k is the value of the keyword TFIELDS) should also be included, and the keyword TTYPEn\ (n=1, 2,, k, where k is the value of the keyword TFIELDS) should be included. Detailed usage of keywords is shown in Appendix A.
Table 6 Mandatory keywords in standard binary extensions #
Last line
Keyword name
XTENSION
BITPIX
NAXISI
NAXIS2
PCOUNT
GCOUNT
TFIELDS
(Other keywords, if keyword TFIELDS is not zero, should also include: TFORMn, n=1,2,,k, where k is the value of keyword TFIELDS should include:
TTYPEn, n=1.2,*,k, where k is the value of keyword TFIELDSEND
Value indicator
Keyword value
BINTABLE
Non-negative integer
Non-negative integer
Non-negative integer not greater than 999|| tt||A string
A string
This keyword has no value
6.2.3.2 Data sequence
6.2.3.2.1 Structure
The structure of the binary table consists of the main data table and an optional heap area. If the last FITS block is not filled, all remaining bits should be filled with binary 0. 6.2.3.2.2 Main data table
GB/T37846—2019
The main data table consists of a two-dimensional byte array, each row has the same number of bytes, and the byte length and number of rows of each row are declared by the keyword "NAXIS1" and the keyword "NAXIS2\ in the unit header respectively. The data area is immediately followed by the last FITS block in the unit header area. The row is located at the beginning of the data area, and each row is stored continuously thereafter, without being restricted by the FITS block structure. In each row, all fields are stored in column order.
Each row in the two-dimensional byte array consists of a series of fields, and the number of fields in each row is declared by the keyword TFIELDS\ in the unit header. The position and format of the fields in each row should be the same, and the number of data and data type of each field are declared by the keyword "TFORMn\ in the unit header. If the value given by the keyword "TFORMn" is 0, the field is empty, see A.1. The format definition of the field is shown in Table A.3, including:
a) Logical type. It is "T" (true) or "F" (false). If the number of bytes in the field is zero, it means that its value is "NULL". Bit type. It is a series of bit numbers, starting from the highest bit and decreasing to the lowest bit. The bit array should store integers b)
bytes. If the number is less than an integer, it will be padded with "O". "NULL" is not defined in the bit array. Character type. It is a string consisting of O or more ASCII texts. It can be terminated by ASCIIc
"NULL" before reaching the length of the repetition number. A string whose first character is ASCIINULL is defined as an empty string. The characters after the first ASCII "NULL" are undefined. Unsigned 8-bit integer type. It is an unsigned 8-bit integer, arranged in positive byte order. The empty value of the field is given by the keyword "TNULLn\d)
16-bit integer type. It is a signed 16-bit integer represented by two's complement, containing two bytes, arranged in positive byte order. The empty value is given by the keyword "e)
"TNULLn"
32-bit integer type. It is a signed 32-bit integer represented by two's complement, containing four bytes, arranged in positive byte order. The null value is given by the keyword "TNULLn\.
64-bit integer. It is a signed 64-bit integer represented in two's complement, consisting of eight bytes, arranged in positive byte order. The null value is given by the keyword "TNULLn\.
Single-precision floating-point number. It is a 32-bit floating-point number that complies with IEEE754-2008, arranged in positive byte order. All IEEE special h)
values can be used. The null value is given by IEEENaN. Double-precision floating-point number. It is a 64-bit floating-point number that complies with IEEE754-2008, arranged in positive byte order. All IEEE special i
values can be used, and the null value is given by IEEENaN. Single-precision complex number. It is a pair of 32-bit single-precision floating-point numbers, in each pair the first value is the real part of the complex number, and the other value is the j)
imaginary part of the complex number. If the real or imaginary part of a complex number is IEEENaN, the entire complex number is a null double-precision complex number. It is a pair of 64-bit double-precision floating-point numbers, where the first value in each pair is the real part of the complex number and the other value is the imaginary part of the complex number. If the real or imaginary part of a complex number is IEEENaN, the entire complex number is a null array indicator. There are two types of array indicators, \P" and \Q\, corresponding to two 32-bit signed integers and two 64-bit signed integers respectively. The first integer represents the length of the storage array, and the second integer represents the offset of the array at the beginning of the heap area. Negative values are undefined. If the storage length of the array is 0, the offset should be set to 0. 6.2.3.2.3 Heap area
The heap area is located after the main data table and is used to store variable arrays. There may be a gap between the heap area and the main data table. The heap area does not require data alignment, and variable arrays can be stored in any order in the heap area. Array indicators with two or more rows can also point to the same storage location.
GB/T37846—2019
6.2.3.3 Variable array
If a variable array exists, it should be declared in the unit header with the keyword \TFORMn\, and its value should be "rPt(emax)" or \rQt(emax)". Among them, \P\ and \Q\ are array indicators, and "r\ can only take the value of \0", "1\ or not appear. "t\ should select the data format in 6.2.3.2.2, but should not include the array indicator. The value of "emax\ should be greater than or equal to the maximum number of \t\ type data stored in each row in the heap area. The array indicator of the variable array is stored in the main data table, and the data of the variable array is stored in the heap area. 7 Special records
Special records should be after the last HDU in the file. The size is an integer number of FITS blocks. Its structure is not restricted by the requirements of the specification extension unit.
The first 8 bytes of the special record should not contain the string "XTENSION\ and the string "SIMPLE". Note: Special records are not recommended for use at present. 8 Keyword records
8.1 Record mode
Keyword records are text composed of 80 bytes of ASCII characters, which are used to describe the keyword name, value indicator, keyword value and comments. The last three items are optional
When the keyword value exists, a value indicator should be added between the keyword name and the keyword value. Unless otherwise specified, keyword records can appear in any order, and should be arranged in order of importance. 8.2 Keyword name
The keyword name should be located at the 1st to 8th bytes of the keyword record, should be left-aligned, and contain 8 characters. If it is insufficient, it should be padded with ASCII spaces. The keyword name should use numbers between "0" and "9", uppercase English characters "A" to "Z", underscores ("_"), and hyphens ("-"). The integer part of the index keyword must not have a leading digit "0". Example: NAXIS1 and NAXIS_1 are keyword names that meet the requirements, and NAXISO01 is a keyword name that does not meet the requirements. The categories and usage of keywords are shown in Appendix A. Mandatory keywords can only appear once in the same file header or unit header. Other keywords with value fields should only appear once. If they appear If the keyword is used twice or more, the value of the keyword is undefined. The keyword name should be selected from the keywords defined in Appendix A. If a new keyword needs to be defined, its name should comply with the requirements of this clause and not conflict with the existing keyword name. It should be stated in the corresponding basic file header or unit header. 8.3 Value indicator
The value indicator should be located at the 9th and 10th bytes of the keyword record. If two ASCII characters = " appear at the 9th and 10th bytes, the keyword name has a value field associated with it, except for the annotative keyword name (see A.2.4). 8.4 Keyword value and comment
The keyword value and comment should be located at bytes 11 to 80 of the keyword record. The keyword value should consist of ASCII text. The value field can be empty, that is, composed entirely of spaces, indicating that the keyword value is undefined. If a comment is added after the keyword value, a slash (\/\) should be added between the value and the comment. Comments can only consist of ASCII text characters. The format of the keyword value includes fixed format and free format. The values of mandatory keywords (see A.1) should be in a fixed format. The values of other keywords should also be in a fixed format. The types and formats of keyword values include: a) String type. A string consisting of ASCII text, with the content in single quotes ("\). If there is a single quote in the string, it must be represented by two consecutive single quotes. The string length should not exceed 68 characters. There is no minimum length requirement, but 8
Tip: This standard content only shows part of the intercepted content of the complete standard. If you need the complete standard, please go to the top to download the complete standard document for free.