Creating a new plugin: 17
CIF files part 2
After working with these files for a while, I've come to the conclusion that extracting the required data from them is not as simple as with the legacy PDB format, but the way the data is structured is much more logical and flexible. On balance, I think I prefer to be working with these files, because the older format was a bit of a mess - just look at the issue with missing residues, where there is no formal way to present these, as an example.
But it isn't all sweetness and light. I ran across an odd problem with one file I looked at. Recall that in a CIF file data items in a record are separated by spaces (or tabs). What happens then if a data item itself contains spaces? See this simple loop_ section from a CIF file:
There are five data items but in all the records the second data item contains spaces. If the record is split by spaces, this would give between 6 and 8 data items, not 5 (which is not allowed, incidentally - the number of data items in each record must always match the number of named items in the loop). Of course, the creators of this file format thought of this, and spaces are allowed in a data item if the item is enclosed in single quotes, as you can see has been done here.
This is all good and my record split function worked perfectly with the single quotes. Now look at this section from a legacy PDB file:
You can see that four of the HETATM records have atom names with a single quote - C1' (C1 prime), C2' (C2 prime) and so on. If you try to split that record taking single quotes as markers to enclose an item with spaces, it won't work. This isn't a problem for legacy PDB files, but it is for CIF files. Here's how it is solved, with the same five lines from the corresponding CIF file:
In this case, the data item containing a single quote has to be enclosed in double quotes. So the split function has to take into account not only single quotes, but double quotes and what to do with single quotes enclosed in double quotes! It all works, but it adds to the complexity of reading these files.
When does a loop section stop?
We've see that a loop section starts with the 'loop_' keyword, is followed by a block of data item names, and that is followed by the data records. But when do the data records for that loop_ section come to an end? There is no 'loop_end_' keyword, as might have been desirable. In all the files I have seen, the data blocks for all loop_ sections end with a comment line (which begins with '#'). But that is entirely optional and is usually included just to improve readability. So when does the data come to an end? It turns out that there are four possibilities:
- another 'loop_' keyword is reached
- a 'data_' keyword is reached
- a line is reached which starts with an underscore, as this must be a standalone data item
- the end of the file is reached
So we need to test for all these eventualities, otherwise lines could be added to the data we are seeking which are nothing to do with the data required and certainly would not parse correctly.
The problem with 'data_'
This is a difficult one. CIF files contain all the information about a protein and any associated molecules within a block starting with the keyword 'data_'. This is often - though it doesn't have to be - the first line in the file. However, CIF files can, perfectly legitimately, contain multiple data_ blocks. These might contain details of different variations in the structure, for example. A data_ block need not contain any atom coordinate data at all, for example it might have details about the authors of a publication and so on. This is a problem, because if we ignore the presence of multiple data_ blocks (and a block ends only when a new 'data_' keyword is encountered) we might include atom data from multiple blocks in one rendered model, which is certainly inaccurate and will probably look very wrong when the model is generated. How should we handle this? I have seen it said that all the CIF files in the protein data bank only contain one data_ block, but I would prefer not to rely on that always being the case.
Some apps simply use the first data_ block (or the first one that contains atom data, anyway) and ignore subsequent ones. If a user wants to see the other blocks, they have to split the file into multiple files before doing so. An alternative would be to let the user choose which data_ block to use, but that would involve pre-scanning the file to determine how many blocks there were and asking the user to pick one. My preferred solution is to use the first data block which contains atom data and ignore the others, but also to include the ability to split multi-block files. Hopefully that would be straightforward, and then the user can choose which file to load.
Bonds between atoms
Bonds in standard amino acid and nucleotide chains are handled in the same way as in a legacy PDB file, but bonds between heterogen atoms are not. In the PDB file, these are found in CONECT records (see section 13 in this series) but there is no direct equivalent in a CIF file. Instead, bond details can be found in two places. The first is the '_struct_conn' section, which includes a data item giving the type of bond, such as a disulphide bond, and from this it is easy to extract disulphide bond data. But for heterogen atoms, bonds between molecules are also found in _struct_conn; an example is show here:
What this does not show are bonds between atoms within non-standard molecules. For this, we can look in the section _chem_comp_bond. For any molecule in the file - such as the amino acids which occur in it, as well as heterogen atoms - this section shows which atoms in the molecule bond to which. In that sense, this actually duplicates the bonds table loaded by the plugin and which also lists the bonds between atoms. The only real difference is that _chem_comp_bond can also include bonds to hydrogen atoms, which to be honest is not that useful if the ATOM records don't include hydrogen atoms (which they almost always don't). An example is shown here, in which a heterogen molecule with identifier 'BGC' has a number of bonds between its atoms. A lot of the data records in this section have been omitted for the sake of clarity:
You can see that this section also contains bond details for standard amino acid residues, including some to hydrogen atoms.
So to generate the bonds between the atoms in a heterogen molecule, or between a heterogen molecule and some other molecule, we have to parse both these sections. From those we can generate the bonds which in a PDB file are specified by the CONECT records. This is more complex, but better in the sense that CONECT records in a PDB file are a mess.
The first group is the _struct_conn category. In the old PDB CONECT records, connections are given using atom serial numbers, which makes it easy to find any atom and get its 3D position. With two atoms, we can create a spline to represent the bond, but _struct_conn records don't do that. Without the atom number, we will need the chain identifier (but see the note at the end of this page), the compound name, the atom name, and the sequence number. We need that last item because there might be more than one molecule of a particular type in the file.
If you look at this section from a CIF file, you can see three of the data records. These are covalent bonds - there are other bond types possible, such as disulphide bonds, but these are all for bonds between heterogen molecules:
We have the 'chain' ID (yellow highlight), the compound name, 'BGC' in this case(green), the atom name (blue) and...where is the sequence ID? The field '_struct_conn.ptnr1_label_seq_id' which should give the residue sequence number only has a dot, indicating that this data is not relevant in this case. But without it, we can't find the correct atom and get its coordinates. It may be - and I would like to think this is the case - that the first record for the BGC compound refers to the first such compound in the HETATM records, the second record for the second instance of the compound, and so on. I've been unable to confirm if that is the case. However, there is another solution. It seems that the reason for the missing data is that the field '_struct_conn.ptnr1_label_seq_id' only applies to amino acid or nucleotide polymers and not to heterogen molecules. The sequence number, however, is assigned by the creator of the file and that is present in the _struct_conn records, in the data items '_struct_conn.ptnr1_auth_seq_id' and '_struct_conn.ptnr2_auth_seq_id'. These are highlighted in red in the above image. So for heterogen molecules, we can ignore the missing data and use the author-assigned fields instead.
Missing residues
This was the final thing implemented when reading PDBs. In CIF files, this is done a lot better because there is a specific data category for this purpose. This is the category '_pdbx_unobs_or_zero_occ_residues'. All the plugin needs to know is the chain with the missing residues and the sequence number of each missing residue. Once we have that, the same code which creates the dashed line as a placeholder for the missing residues can be used for CIF files, too.
Is that it?
I think so. That should complete the implementation of CIF files to the equivalent of the legacy PDB files. Of course, there may be more features to add which occur to me. It's also true that both PDB and CIF files contain a lot more data than I have used so far. It's at this point, though, that my knowledge of macromolecule structure comes to an end, and whether any of the remaining data is relevant to visualisation of these molecules is currently unclear to me.
What I think is clear is that this plugin produces a protein visualisation that is accurate - as far I can tell - and complete in many of the major areas. If more is needed, it will have to wait until the plugin is released and users (if there are any) feed back what isn't correct and what else they'd like to see in it. For my part, what I need to do now is test it extensively, fix any bugs, make sure that error checking is as complete as I can make it, and write some documentation. Then it will be (finally) ready for release.
This will be the final update to this series. There won't be any more until the plugin is released. Hopefully that won't be too long!
Note: the 'chain' identifier is a term I have used for heterogen atoms but is not actually correct. Chains should really only be used for amino acid or nucleotide polymers, not for heterogen atoms. In CIF files, the correct term for all 'chains' is 'asymmetric unit' which is why these files use data names such as '_struct_conn.ptnr1_label_asym_id'. I've used 'chain' because I've got used to it and the meaning of 'asymmetric unit' is not immediately obvious.
Page last updated August 8th 2026