The GenBank sequence database is an annotated collection of all publicly available nucleotide sequences and their protein translations. This database is produced at National Center for Biotechnology Information (
Direct submissions are made to GenBank using
Initially, GenBank was built and maintained at Los Alamos National Laboratory (
In the mid-1990s, the GenBank database became part of the International Nucleotide Sequence Database Collaboration with the EMBL database (
The Collaboration created a
When scientists submit data to GenBank, they have the opportunity to keep their data confidential for a specified period of time. This helps to allay concerns that the availability of their data in GenBank before publication may compromise their work. When the article containing the citation of the sequence or its Accession number is published, the sequence record is released. The database staff request that submitters notify GenBank of the date of publication so that the sequence can be released without delay. The request to release should be sent to
The typical GenBank submission consists of a single, contiguous stretch of DNA or RNA sequence with annotations. The annotations are meant to provide an adequate representation of the biological information in the record. The GenBank
Currently, only nucleotide sequences are accepted for direct submission to GenBank. These include mRNA sequences with coding regions, fragments of genomic DNA with a single gene or multiple genes, and ribosomal RNA gene clusters. If part of the nucleotide sequence encodes a protein, a conceptual translation, called a
Multiple sequences can be submitted together. Such batch submissions of non-related sequences may be processed together but will be displayed in Entrez (
What defines a set? Environmental sample, population, phylogenetic, and mutation sets all contain a group of sequences that spans the same gene or region of the genome. Environmental samples are derived from a group of unclassified or unknown organisms. A population set contains sequences from different isolates of the same organism. A phylogenetic set contains sequences from different organisms that are used to determine the phylogenetic relationship between them. Sequencing multiple mutations within a single gene gives rise to a mutation set.
All sets, except segmented sets, may contain an alignment of the sequences within them and might include external sequences already present in the database. In fact, the submitter can begin with an existing alignment to create a submission to the database using the Sequin submission tool. Currently, Sequin accepts
Segmented sets are a collection of noncontiguous sequences that cover a specified genetic region. The most common example is a set of genomic sequences containing
Diagram showing the orientation and gaps that might be expected in high-throughput sequence from phases 1, 2, and 3.
Phase 0, 1, and 2 records are in the HTG division of GenBank, whereas phase 3 entries go into the taxonomic division of the organism, for example, PRI (primate) for human. An entry keeps its Accession number as it progresses from one phase to another but receives a new Accession.Version number and a new gi number each time there is a sequence change.
To submit sequences in bulk to the HTG processing system, a center or group must set up an FTP account by writing to
fa2htgs is a command-line program that is downloaded to the user's computer. The submitter invokes a script with a series of parameters (arguments) to create a submission. It has an advantage over Sequin in that it can be set up by the user to create submissions in bulk from multiple files.
Submissions to HTG must contain three identifiers that are used to track each HTG record: the genome center tag, the sequence name, and the Accession number. The genome center tag is assigned by NCBI and is generally the FTP account login name. The sequence name is a unique identifier that is assigned by the submitter to a particular clone or entry and must be unique within the group's submissions. When a sequence is first submitted, it has only a sequence name and genome center tag; the Accession number is assigned during processing. All updates to that entry must include the center tag, sequence name, and Accession number, or processing will fail.
Submitters deposit HTGS sequences in the form of Seq-submit files generated by Sequin, fa2htgs, or their own
Entries can fail HTG processing because of three types of problems:
Formatting: submissions are not in the proper Seq-submit format. Identification: submissions may be missing the genome center tag,
sequence name, or Accession number, or this information is incorrect. Data: submissions have problems with the data and therefore fail
the validator checks.
When submissions fail HTG processing, a GenBank annotator sends email to the sequencing center, describing the problem and asking the center to submit a corrected entry. Annotators do not fix incorrect submissions; this ensures that the staff of the submitting genome center fixes the problems in their database as well.
The processing pathway also generates reports. For successful submissions, two files are generated: one contains the submission in GenBank flat file format (without the sequence); and another is a status report file. The status report file, ac4htgs, contains the genome center, sequence name, Accession number, phase, create date, and update date for the submission. Submissions that fail processing receive an error file with a short description of the error(s) that prevented processing. The GenBank annotator also sends email to the submitter, explaining the errors in further detail.
When successful submissions are loaded into GenBank, they undergo additional validation checks. If GenBank annotators find errors, they write to the submitters, asking them to fix these errors and submit an update.
Genome centers are taking multiple approaches to sequencing complete genomes from a number of organisms. In addition to the traditional clone-based sequencing whose data are being submitted to HTGS, these centers are also using a
Each sequencing project is assigned a stable project ID, which is made up of four letters. The Accession number for a WGS sequence contains the project ID, a two-digit version number, and six digits for the contig ID. For instance, a project would be assigned an Accession number AAAX00000000. The first assembly version would be AAAX01000000. The last six digits of this ID identify individual contigs. A master record for each assembly is created. This master record contains information that is common among all records of the sequencing project, such as the biological source, submitter, and publication information. There is also a link to the range of Accession numbers for the individual contigs in this assembly.
WGS submissions can be created using tbl12asn, a utility that is packaged with the Sequin submission software. Information on submitting these sequences can be found at
Expressed Sequence Tags (
EST, STS, and GSS sequences reside in their respective divisions within GenBank, rather than in the taxonomic division of the organism. The sequences are maintained within GenBank in the dbEST, dbSTS, and dbGSS databases.
Because of the large numbers of sequences that are submitted at once, dbEST, dbSTS, and dbGSS entries are stored in relational databases where information that is common to all sequences can be shared. Submissions consist of several files containing the common information, plus a file of the sequences themselves. The three types of submissions have different requirements, but all include a Publication file and a Contact file. See the
In general, users generate the appropriate files for the submission type and then email the files to
HTC records are High-Throughput cDNA/mRNA submissions that are similar to ESTs but often contain more information. For example, HTC entries often have a systematic gene name (not necessarily an official gene name) that is related to the lab or center that submitted them, and the longest open reading frame is often annotated as a coding region.
FLIC records, Full-Length Insert cDNA, contain the entire sequence of a cloned cDNA/mRNA. Therefore, FLICs are generally longer, and sometimes even full-length, mRNAs. They are usually annotated with genes and coding regions, although these may be lab systematic names rather than functional names.
HTC entries are usually generated with
HTC entries undergo the same validation and processing as non-bulk submissions. Once processing is complete, the records are loaded into GenBank and are available in Entrez and other retrieval systems.
FLICs are processed via an automated FLIC processing system that is based on the HTG automated processing system. Submitters use the program tbl2asn to generate their submissions. As with HTG submissions, submissions to the automated FLIC processing system must contain three identifiers: the genome center tag, the sequence name (SeqId), and the Accession number. The genome center tag is assigned by NCBI and is generally the FTP account login name. The sequence name is a unique identifier that is assigned by the submitter to a particular clone or entry and must be unique within the group's FLIC submissions. When a sequence is first submitted, it has only a sequence name and genome center tag; the Accession number is assigned during processing. All updates to that entry include the center tag, sequence name, and Accession number, or processing will fail.
The FLIC processing system is analogous to the HTG processing system. Submitters deposit their submissions in the FLICSEQSUBMIT directory of their FTP account and notify us that the submissions are there. We then run the scripts to pick up the files from the FTP site and copy them to the processing pathway, as well as to an archive. Once processing is complete and if there are no errors in the submission, the files are automatically loaded into GenBank.
As with HTG submissions, FLIC entries can fail for three reasons: problems with the format, problems with the identification of the record (the genome center, the SeqId, or the Accession number), or problems with the data itself. When submissions fail FLIC processing, a GenBank annotator sends email to the sequencing center, describing the problem and asking the center to submit a corrected entry. Annotators do not fix incorrect submissions; this ensures that the staff of the submitting genome center fixes the problems in their database as well. At the completion of processing, reports are generated and deposited in the submitter's FTP account, as described for HTG submissions.
Direct submissions to GenBank are prepared using one of two submission tools, BankIt or Sequin.
Completed Sequin submissions should be emailed to GenBank at
All direct submissions to GenBank, created either by Sequin or BankIt, are processed by the GenBank annotation staff. The first step in processing submissions is called triage. Within 48 hours of receipt, the database staff reviews the submission to determine whether it meets the minimal criteria for incorporation into GenBank and then assigns an Accession number to each sequence. All sequences must be >50 bp in length and be sequenced by, or on behalf of, the group submitting the sequence. GenBank will not accept sequences constructed
Triaged submissions are subjected to a thorough examination, referred to as the indexing phase. Here, entries are checked for:
Biological validity. For example, does the conceptual translation of a coding region match the amino acid sequence provided by the submitter? Annotators also ensure that the source organism name and lineage are present, and that they are represented in NCBI's taxonomy database. If either of these is not true, the submitter is asked to correct the problem. Entries are also subjected to a series of BLAST similarity searches to compare the annotation with existing sequences in GenBank. Vector contamination. Entries are screened against NCBI's Publication status. If there is a published citation, Formatting and spelling. If there are problems with the sequence or annotation, the annotator works with the submitter to correct them.
Completed entries are sent to the submitter for a final review before release into the public database. If the submitters requested that their sequences be released after processing, they have 5 days to make changes prior to release. The submitter may also request that GenBank hold their sequence until a future date. The sequence must become publicly available once the Accession number or the sequence has been published. The GenBank annotation staff currently processes about 1,900 submissions per month, corresponding to approximately 20,000 sequences.
GenBank annotation staff must also respond to email inquiries that arrive at the rate of approximately 200 per day. These exchanges address a range of topics including:
updates to existing GenBank records, such as new annotation or sequence changes problem resolution during the indexing phase requests for release of the submitter's sequence data or an extension of the hold date requests for release of sequences that have been published but are not yet available in GenBank lists of Accession numbers that are due to appear in upcoming issues of a publisher's journals reports of potential annotation problems with entries in the public database requests for information on how to submit data to GenBank
One annotator is responsible for handling all email received in a 24-hour period, and all messages must be acted upon and replied to in a timely fashion. Replies to previous emails are forwarded to the appropriate annotator.
The annotation staff uses a variety of tools to process and update sequence submissions. Sequence records are edited with Sequin, which allows staff to annotate large sets of records by global editing rather than changing each record individually. This is truly a time saver because more than 100 entries can be edited in a single step (see
The GenBank direct submissions group has processed more than 50 complete microbial genomes since 1996. These genomes are relatively small in size compared with their eukaryotic counterparts, ranging from five hundred thousand to five million bases. Nonetheless, these genomes can contain thousands of genes, coding regions, and structural RNAs; therefore, processing and presenting them correctly is a challenge. Currently, the DDBJ/EMBL/GenBank Nucleotide Sequence Database Collaboration has a 350-kilobase (kb) upper size limit for sequence entries. Because a complete bacterial genome is larger than this arbitrary limit, it must be split into pieces. GenBank routinely splits complete microbial genomes into 10-kb pieces with a 60-bp overlap between pieces. Each piece contains approximately 10 genes. A CON entry, containing instructions on how to put the pieces back together, is also made. The CON entry contains descriptor information, such as source organism and references, as well as a join statement providing explicit instructions on how to generate the complete genome from the pieces. The Accession number assigned to the CON record is also added as a secondary Accession number on each of the pieces that make up the complete genome (see The information toward the
Submitters of complete genomes are encouraged to contact us at
Complete genome submissions are reviewed by a member of the GenBank annotation staff to ensure that the annotation and gene and protein identifiers are correct, and that the entry is in proper GenBank format. Any problems with the entry are resolved through communication with the submitter. Once the record is complete, the genome is carefully split into its component pieces. The genome is split so that none of the breaks occurs within a gene or coding region. A member of the annotation staff performs quality assurance checks on the set of genome pieces to ensure that they are correct and representative of the complete genome. The pieces are then loaded into GenBank, and the CON record is created.
The microbial genome records in GenBank are the building blocks for the
The vast amount of publicly available data from the human genome project and other genome sequencing efforts is a valuable resource for
scientists throughout the world. A laboratory studying a particular gene
or gene family may have sequenced numerous cDNAs but has neither the
resources nor inclination to sequence large genomic regions containing
the genes, especially when the sequence is available in public
databases. The researcher might choose then to download genomic
sequences from GenBank and perform experimental analyses on these
sequences. However, because this researcher did not perform the
sequencing, the sequence, with its new annotations, cannot be submitted
to DDBJ/EMBL/GenBank. This is unfortunate because important scientific
information is being excluded from the public databases. To address this
problem, the International Nucleotide Sequence Database Collaboration
established a separate section of the database for such TPA (see
All sequences in the TPA database are derived from the publicly available collection of sequences in DDBJ/EMBL/GenBank. Researchers can submit both new and alternative annotations of genomic sequence to GenBank. TPA entries can be also created by combining the exon sequences from genomic sequences or by making contigs of EST sequences to make mRNA sequences. TPA submissions must use sequence data that are already represented in DDBJ/EMBL/GenBank, have annotation that is experimentally supported, and appear in a peer-reviewed scientific journal. TPA sequences will be released to the public database only when their Accession numbers and/or sequence data appear in a peer-reviewed publication in a biological journal.
PubMed is a database developed by the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM), one of the institutes of the National Institutes of Health (NIH). The database was designed to provide access to citations (with abstracts) from biomedical journals. Subsequently, a linking feature was added to provide access to full-text journal articles at Web sites of participating publishers, as well as to other related Web resources. PubMed is the bibliographic component of the NCBI's Entrez retrieval system.
In addition to MEDLINE citations, PubMed® provides access to non-MEDLINE resources, such as out-of-scope citations, citations that precede MEDLINE selection, and PubMed Central (PMC; see
In response to new approaches to electronic publishing, PubMed can now also accommodate articles published electronically in advance of being collected into an issue. We refer to these citations as “ahead of print” or “epub” citations.
All content in PubMed ultimately comes from publishers of biomedical journals, and journals that are to be included in MEDLINE are subject to a selection process. The
Electronic data submission benefits everyone: publishers, the NLM, and users. For the NLM, it eliminates the tremendous costs associated with entering data by hand. For publishers and users, it means that newly published data appear rapidly and accurately in PubMed. Some publishers are now making pre-publication material available before it is formally published (“ahead of print” or “epub” citations); others are publishing electronic-only journals. By close collaboration with the publisher, the citations for these publications can appear in PubMed on the same day as the article is published.
Furthermore, electronic data submission allows publishers to create links from abstracts in PubMed to the full text of the appropriate articles available on their own Web site. This can be achieved using LinkOut (
Although the NLM works with many publishers directly, some publishers contract with commercial data aggregators, companies that prepare and submit the publisher's data to the NLM. Many aggregators also host publisher data on their Web sites.
All electronic data are supplied via
NCBI staff will guide new data providers through the approval process for file submission. New providers are asked to submit test files, which are then checked for XML formatting and syntax and for bibliographic accuracy and completeness. The files are revised and resubmitted as many times as necessary until all criteria are met. Once approved, a private account is set up on our FTP site to receive new journal issues, or in the case of online publications, individual articles as they are added to the publisher's Web site. We run a file-loading script that automatically processes the files daily, Monday through Friday at approximately 9:00 a.m. (Eastern Time). The new citations are assigned a PubMed ID number (PMID), a confirmation report is sent to the provider, and the new citations usually become available in PubMed sometime after 11:00 a.m. the next day, Tuesday through Saturday.
After posting in PubMed, the citations are forwarded to NLM's Indexing Section for bibliographic data verification and for the addition of subject indexing terms from Medical Subject Headings [
PubMed is one of the NCBI databases within the relational database management system,
Requests for NCBI services, including PubMed, are first proxied through three load-balanced Dell PowerEdge 1650 servers, each with two central processing units. The proxy servers, in turn, load-balance requests forwarded on to the Web servers for PubMed and other NCBI services.
The PubMed Web servers comprise eight Dell PowerEdge 8450 servers. The Dell servers have eight central processing units, 8 GB of memory, and about 300 GB of disk space and run the Linux operating system.
The Web servers retrieve PubMed records from two Sybase SQL database servers, which run on Sun Enterprise 450s. To accommodate the data volume output by PubMed and other Web-based services, the NLM has a high-speed connection (OC-3, up to 155 Mbits/sec) to the Internet, as well as a 622 Mbits/sec connection (OC-12) to Internet2, the noncommercial network used by many leading research universities.
Citations in PubMed are assigned one of three citation status tags that display next to the PubMed ID (PMID) numbers on all PubMed citations. The citation status tags indicate the citation's stage in the MEDLINE indexing process. The three tags are:
Most citations that are received electronically from publishers progress through “in process” status to MEDLINE status. Those citations not indexed for MEDLINE remain tagged [PubMed - as supplied by publisher]. Citations with “in process” status proceed to MEDLINE status after MeSH terms, publication types, sequence Accession numbers, and other indexing data are added.
All records are added to PubMed Monday through Friday and become available for viewing Tuesday through Saturday. For additional information, please see the NLM
The aim of the computer indexing process is to automatically create multiple machine-readable access points that refer to the different components of the journal citations for use when searching PubMed. The citations are loaded into PubMed from both the NLM Data Creation and Maintenance System (DCMS) and directly from journal publishers (
During the computer indexing process, the citation information is broken down into index fields such as Journal Name, Author Name, and Title/Abstract. The words in each of the fields are checked against the corresponding index (i.e., title words in a new citation are looked up in the Title/Abstract Index). If the word already exists, the PMID of the citation is listed with that index term. If the word is a new one for the Index, it is added as a new Index term, and the PMID is listed alongside it. (In the first instance that the term already exists, the new term will have only this one citation associated with it; this is how the PubMed indexes grow.)
Each PubMed citation is, therefore, associated with several indexes, and in cases similar to the Title/Abstract Index, many different index terms can refer back to a single citation. Likewise, commonly used terms will refer to thousands of citations (the term “cell”, for example, is found in the Title/Abstract of 1,092,124 citations at the time of this writing). The Field Indexes can be browsed by using PubMed's
PubMed uses MeSH Translation Table Journals Translation Table 3. Author Index
The MeSH Terms
See-Reference mappings (also known as entry terms) for MeSH terms Mappings derived from the Unified Medical Language System ( Names of Substances and synonyms to the Names of Substances (now known as Supplementary Concept Substance Names)
If the search term is found in this translation table, the term will be mapped to the appropriate MeSH term, and the Indexes will be searched as both the text word entered by the user and the MeSH term:
“Gallstones” is an entry term for the MeSH term “cholelithiasis” in the MeSH translation table. Search translated to: “cholelithiasis” [MeSH Terms] OR gallstones [Text Word]
When a term is searched as a MeSH term, PubMed automatically searches that term plus the more specific terms underneath in the “Breast cancer” is an entry term for the MeSH term “breast neoplasms” in the MeSH translation table. “Breast neoplasms” has the specific headings “breast neoplasms, male”, “mammary neoplasms”, “mammary neoplasms, experimental”, and “phyllodes tumor”, all of which are also searched.
If the search term(s) is not found in the MeSH Translation Table, the PubMed search algorithm then looks up the term in the “New England Journal of Medicine” maps to N Engl J Med. Search translated to: “N Engl J Med” [Journal Name]
If a journal name is also a MeSH term, PubMed will search the term as both a MeSH term and as a Text Word, but not as a Search translated as: “cells” [MeSH Terms] OR cell [Text Word] Search translated as: “Cell” [Journal]
If the phrase is not found in MeSH or the Journals Translation Table and is a word with one or two letters after it, PubMed then checks the
If only one initial is used, PubMed finds all names with that first initial, and if only an author's last name is entered, PubMed will search that name in All Fields. It will not default to the Author Index because an initial does not follow the last name:
Search translated as: o'malley fa, o'malley fb, o'malley fc, o'malley fd, o'malley f jr, etc. Search translated as: “o'malley” [All Fields]
A history of the NLM's author indexing policy regarding the number of authors to include in a citation is outlined in
| Dates | Policy |
|---|---|
| 1966–1984 | MEDLINE did not limit the number of authors. |
| 1984–1995 | The NLM limited the number of authors to 10, with “et al.” as the eleventh occurrence. |
| 1996–1999 | The NLM increased the limit from 10 to 25. If there were more than 25 authors, the first 24 were listed, the last author was used as the 25th, and the twenty-sixth and beyond became “et al.”. |
| 2000–present | MEDLINE does not limit the number of authors. |
It is possible to override PubMed's Automatic Term Mapping by using search rules, syntax, and qualifying terms with search field abbreviations.
The
Search term: o'malley [au] will search only the author field. Specifying the field precludes the Automatic Term Mapping, which would result in the search o'malley[All Fields] if the field were not specified. Similarly, using the search term Cell [Journal] avoids using the MeSH Translation Table, which would interpret Cell as only a text word and MeSH term.
A simple search can be conducted from the
If more than one term is entered in the query box, PubMed will go through the Automatic Term Mapping protocol described in the previous section, first looking for all the terms, as typed, to find an exact match. If the exact phrase is not found, PubMed clips a term off the end and repeats Automatic Term Mapping, again looking for an exact match, but this time to the abbreviated query. This continues until none of the words are found in any one of the translation tables. In this case, PubMed combines terms (with the AND Boolean operator) and applies the Automatic Term Mapping process to each individual word. PubMed ignores
Translated as: ((“ascorbic acid” [MeSH Terms] OR vitamin c [Text Word]) AND (“common cold” [MeSH Terms] OR common cold [Text Word]))
Translated as: (((“single person” [MeSH Terms] OR single [Text Word]) AND (“cell separation” [MeSH Terms] OR cell separation [Text Word])) AND (“brain” [MeSH Terms] OR brain [Text Word]))
If a phrase of more than two terms is not found in any translation table, then the last word of the phrase is dropped, and the remainder of the phrase is sent through the entire process again. This continues, removing one word at a time, until a match is found.
If there is no match found during the Automatic Term Mapping process, the individual terms will be combined with AND and searched in All Fields.
One can see how PubMed interpreted a search by selecting
There are a variety of ways that PubMed can be searched in a more sophisticated manner than simply typing search terms into the search box and selecting
The following resources are available to facilitate effective searches:
PubMed retrieves and displays search results in the Summary format in the order the record was initially added to PubMed, with the most recent first. (Note that this date can differ widely from the publication date.) Citations can be viewed in several other
A variety of links can be found on PubMed citations including:
NCBI databases, as well as other resources, may be available from the
The Entrez system provides three distinct ways to create Web
The Entrez Programming Utilities can be used to create URL links directly to all Entrez data, including PubMed citations and their link information, without using the front-end Entrez query engine. These Utilities provide a fast, efficient way to search and download citation data.
If you need more assistance, please contact our
Additional information is also available in the
The resources provided by NCBI for studying the three-dimensional (3D) structures of proteins center around two databases: the Molecular Modeling Database (MMDB), which provides structural information about individual proteins; and the Conserved Domain Database (CDD), which provides a directory of sequence and structure alignments representing conserved functional domains within proteins(
To enable scientists to accomplish these tasks, NCBI has integrated MMDB and CDD into the Entrez retrieval system (
Protein structures can be visualized using Cn3D, an interactive 3D graphic modeling tool. Details of the structure, such as ligand-binding sites, can be scrutinized and highlighted. Cn3D can also display multiple sequence alignments based on sequence and/or structural similarity among related sequences, 3D domains, or members of a CDD family. Cn3D images and alignments can be manipulated easily and exported to other applications for presentation or further analysis.
The Structure This page can be found by selecting the
Often used in conjunction with
The structures within MMDB are now being linked to the NCBI Taxonomy database (
The second database within the
All the above databases and tools are discussed in more detail in other parts of this Chapter, including tips on how to make the best use of them.
MMDB records have several types of Accession numbers associated with them, representing the following data types:
Each MMDB record has at least three Accession numbers: the PDB code of the corresponding PDB record (e.g., 1CYO, 1B8G); a unique MMDB-ID (e.g., 645, 12342); and a gi number for each protein chain. A new MMDB-ID is assigned whenever PDB updates either the sequence or coordinates of a structure record, even if the PDB code is retained. If an MMDB record contains more than one polypeptide or nucleotide chain, each chain in the MMDB record is assigned an Accession number in Entrez Protein or Nucleotide consisting of the PDB code followed by the letter designating that chain (e.g., 1B8GA, 3TATB, 1MUHB). Each 3D Domain identified in an MMDB record is assigned
a unique integer identifier that is appended to the Accession number of the chain to which it belongs (e.g., 1B8G A 2). This new Accession number becomes its identifier in Entrez 3D Domains. New 3D Domain identifiers are assigned whenever a new MMDB-ID is assigned. For conserved domains, the Accession number is based on
the source database:
To build MMDB (
The data are converted into
After initial processing, 3D domains are automatically identified within each MMDB record. 3D domains are annotations on individual MMDB structures that define the boundaries of compact substructures contained within them. In this way, they are similar to secondary structure annotations that define the boundaries of helical or β-strand substructures. Because proteins are often similar at the level of domains, VAST compares each 3D domain to every other one and to complete polypeptide chains. The results are stored in Entrez as a
To identify 3D domains within a polypeptide chain, MMDB's domain parser searches for one or more breakpoints in the structure. These breakpoints fall between major secondary structure elements such that the ratio of intra- to interdomain contacts remains above a set threshold. The 3D domains identified in this way provide a means to both increase the sensitivity of structure neighbor calculations and also present 3D superpositions based on compact domains as well as on complete polypeptide chains. They are not intended to represent domains identified by comparative sequence and structure analysis, nor do they represent modules that recur in related proteins, although there is often good agreement between domain boundaries identified by these methods.
After initially processing the PDB record, structure staff add a number of links and other information that further integrate the MMDB record with other NCBI resources. To begin, the sequence information extracted from the PDB record is entered into the Entrez Protein and/or Nucleotide databases as appropriate, providing a means to retrieve the structure information from sequence searches. As with all sequences in Entrez, precomputed BLAST searches are then performed on these sequences, linking them to other molecules of similar sequence. For proteins, these BLAST neighbors may be different than those determined by VAST; whereas VAST uses a conservative significance threshold, the structural similarities it detects often represent remote relationships not detectable by sequence comparison. The literature citations in the PDB record are linked to PubMed so that Entrez searches can allow access to the original descriptions of the structure determinations. Finally, semiautomatic processing of the “source” field of the PDB record provides links to the NCBI Taxonomy database. Although these links normally follow the genus and species information given, in some cases this information is either absent in the PDB record or refers only to how a sample was obtained. In these cases, the staff manually enters the appropriate taxonomy links.
The Structure Summary page for each MMDB record summarizes the database content for that record and serves as a starting point for analyzing the record using the NCBI structure tools ( The page consists of three parts: the header, the view bar, and the graphic display. The header contains basic identifying information about the record: a description of the protein (
Although VAST itself is not a database, the VAST results computed for each MMDB record are stored with this record and are summarized on a separate page for the whole polypeptide chain as well as for each 3D domain found in the protein ( The
The non-redundant PDB database (nr-PDB) is a collection of four sets of sequence-dissimilar cluster PDB polypeptide chains assembled by NCBI Structure staff. The four sets differ only in their respective levels of non-redundancy. The staff assembles each set by comparing all the chains available from PDB with each other using the BLAST algorithm. The chains are then clustered into groups of similar sequence using a single-linkage clustering procedure. Chains within a sequence-similar group are automatically ranked according to the quality of their structural data. Details of the measures used to determine structure precision and completeness and the methodology of assembling the nr-PDB clusters can be found on the nr-PDB
CDs are recurring units in polypeptide chains (sequence and structure motifs), the extents of which can be determined by comparative analysis. Molecular evolution uses such domains as building blocks and these may be recombined in different arrangements to make different proteins with different functions. The CDD contains sequence alignments that define the features that are conserved within each domain family. Therefore, the CDD serves as a classification resource that groups proteins based on the presence of these predefined domains. CDD entries often name the domain family and describe the role of conserved residues in binding or catalysis. Conserved domains are displayed in MMDB Structure summaries and link to a sequence alignment showing other proteins in which the domain is conserved, which may provide clues on protein function.
The collections of domain alignments in the CDD are imported either from two databases outside of the NCBI, named Pfam (
Once imported and constructed, each domain alignment in CDD is used to calculate a model sequence, called a
RPS-BLAST (
Analogous to the Structure Summary page, the CD Summary page displays the available information about a given CD and offers various links for either viewing the CD alignment or initiating further searches ( The
In 2002, NCBI released the first group of curated CD records, a new and expanding set of annotated protein multiple sequence alignments and corresponding structure alignments. These new records have Accession numbers beginning with “cd” and have been added to the default CD-Search database. Most curated CD records are based on existing family descriptions from SMART and Pfam, but the alignments may have been revised extensively by quantitatively using three-dimensional structures and by re-examining the domain extent. In addition, CDD curators annotate conserved functional residues, ligands, and co-factors contained within the structures. They also record evidence for these sites as pointers to relevant literature or to three-dimensional structures exemplifying their properties. These annotations may be viewed using Cn3D and thus provide a direct way of visualizing functional properties of a protein domain in the context of its three-dimensional structure. (See
The term “domain” refers in general to a distinct functional and/or structural unit of a protein. Each polypeptide chain in MMDB is analyzed for the presence of two classes of domains, and it is important for users to understand the difference between them. One class, called 3D Domains, is based solely on similar, compact substructures, whereas the second class, called Conserved Domains (CDs), is based solely on conserved sequence motifs. These two classifications often agree, because the compact substructures within a protein often correspond to domains joined by recombination in the evolutionary history of a protein. Note that CD links can be identified even when no 3D structures within a family are known. Moreover, 3D Domain links may also indicate relationships either to structures not included in CDD entries or to structures so distantly related that no significant similarity can be found by sequence comparisons.
Suppose that we are interested in the biosynthesis of aminocyclopropanes and would like to find structural information on important active site residues in any available aminocyclopropane synthases. To begin, we would go to the
Once we have found the Structure Summary page, viewing the 3D structure is straightforward. To view the structure in Cn3D, we simply select the
Upon inspecting the structure, we immediately notice that a small molecule is bound to the protein, likely at the active site of the enzyme. How do we find out what that molecule is? One easy way is to return to the Structure Summary page and select the link to the PDB code, which takes us to the PDB Structure Explorer page for 1B8G. Quickly, we see that pyridoxal-5′-phosphate (PLP) is a HET group, or heterogen, in the structure. Our interest piqued, we would now like to know more about the structural domain containing the active site. Returning to Cn3D, we manipulate the structure so that PLP is easily visible and then use the mouse to double-click on any PLP atom. The molecule becomes selected and turns yellow. Now from the
Given that this enzyme is a dimer, we arbitrarily choose domain 3 in chain A, the accession of which is thus 1B8GA3. By clicking on the 3D Domain bar at a point within domain 3, we are taken to the VAST Structure Neighbors page for this domain, where we find nearly 200 structure neighbors.
Perhaps we would now like to identify some of the most evolutionarily distant structure neighbors of domain 1B8GA3 as a means of finding conserved residues that may be associated with its binding and/or catalytic function. One powerful way of doing this is to choose structure neighbors from phylogenetically distant organisms. We therefore need to combine our present search with a Taxonomy search. Given that 1B8G is derived from the superkingdom Eukaryota, we would like to find structure neighbors in other superkingdom taxa, such as Eubacteria and Archaea. Returning to the Structure Summary page, select the 3D Domains link in the graphic display to open the list of 3D Domains in Entrez. Finding 1B8GA3 in the list, selecting the
Looking at the Archaea results, we find among them 1DJUA3, a domain from an aromatic aminotransferase from
Returning to the VAST Structure Neighbors page for 1B8GA3, we want to select 1DJUA3 and 3TATA2 to display in a structural alignment. One way to do this is to enter these two Accession numbers in the
Cn3D again displays the aligned residues in red, and we can highlight these further by selecting The backbone atoms of the aligned residues of the three structures are shown colored according to their sequence conservation of each position in the alignment. Highly conserved positions are colored more
For an example query on finding and viewing structures, see
To determine the overall shape and size of a protein
To locate a residue of interest in the overall structure
To locate residues in close proximity to a residue of interest
To develop or test chemical hypotheses regarding an enzyme mechanism
To locate or predict possible binding sites of a ligand
To interpret mutation studies
To find areas of positive or negative charge on the protein surface
To locate particularly hydrophobic or hydrophilic regions of a protein
To infer the 3D structure and related properties of a protein with unknown structure from the structure of a
To study evolutionary processes at the level of molecular structure
To study the function of a protein
To study the molecular basis of disease and design novel treatments
The first step to any structural analysis at NCBI is to find the structure records for the protein of interest or for proteins similar to it. One may search MMDB directly by entering search terms such as PDB code, protein name, author, or journal in the Entrez Structure
By using the full array of Entrez search tools, the resulting list of MMDB records can be honed, ideally, to a workable list from which a record can be selected. Users should note that multiple records may exist for a given protein, reflecting different experimental techniques, conditions, and the presence or absence of various ligands or metal ions. Records may also contain different fragments of the full-length molecule. In addition, many structures of mutant proteins are also available. The PDB record for a given structure generally contains some description of the experimental conditions under which the structure was determined, and this file can be accessed by selecting the PDB code link at the top of the Structure Summary page.
Structure Summary pages can also be found from the following NCBI databases and tools:
Select the Structure Select the Choose the PDB database from a Select the From the results of any protein BLAST search, click on a red 'S' linkout to view the sequence alignment with a structure record.
The 3D domains of a protein are displayed on the Structure Summary page. It is useful to know how many 3D domains a protein contains and whether they are continuous in sequence when viewing the full 3D structure of the molecule.
Knowing the secondary structure of a protein can also be a useful prelude to viewing the 3D structure of the molecule. The secondary structure can be viewed easily by first selecting the
Cn3D is a software package for displaying 3D structures of proteins. Once it has been
For an example query on finding and viewing structure neighbors, see
To determine structurally conserved regions in a protein family
To locate the structural equivalent of a residue of interest in another related protein
To gain insights into the allowable structural variability in a particular protein family
To develop or test chemical hypotheses regarding an enzyme mechanism
To predict possible binding sites of a ligand from the location of a binding site in a related protein
To identify sites where conformational changes are concentrated
To interpret mutation studies
To find areas of conserved positive or negative charge on the protein surface
To locate conserved hydrophobic or hydrophilic regions of a protein
To identify evolutionary relationships across protein families
To identify functionally equivalent proteins with little or no sequence conservation
The Vector Alignment Search Tool (VAST) is used to calculate similar structures on each protein contained in the MMDB. The graphic display on each Structure Summary page ( The 3D Domains link transfers the user to Entrez 3D Domains, showing a list of the VAST neighbors. Selecting the chain bar displays the VAST Structure Neighbors page for the entire chain. Selecting a 3D Domain bar displays the VAST Structure Neighbors page for the selected domain.
From any Entrez search, select
A graphic 2D HTML alignment of VAST neighbors can be viewed as follows:
On the lower portion of the VAST Structure Neighbors page ( On the Select
Alignments of VAST structure neighbors can be viewed as a 3D image using Cn3D.
On the lower portion of the VAST Structure Neighbors page (
On the
Select
Cn3D automatically launches and displays the aligned structures. Each displayed chain has a unique color; however, the portions of the structures involved in the alignment are shown in red. These same colors are also reflected in the Sequence/Alignment Viewer. Among the many viewing options provided by Cn3D, of particular use is the
Suppose that we are interested in topoisomerase enzymes and would like to find human topoisomerases that most closely resemble those found in eubacteria and thus may share a common ancestor. Further suppose that through a colleague, we are aware of a recent and particularly interesting crystal structure of a topoisomerase from
The Struture summary page displays only the CDs that give the best match to the protein sequence. To see all of the matching CDs, we can easily perform a full CD-Search. Select the
We now would like to find human proteins that have these same CDs. To perform a CDART search, simply select the
We now would like to view the alignment of the topoisomerase in the human protein to other members of this CD. On the CD-Search page, select the colored bar of this CD to see a CD-Browser window displaying the alignment. Because this is a curated CD record, we are able to view functional features of the protein domain on a structural template. The rightmost menu in the View Alignment bar shows the available features for this domain, whereas the topmost row in the alignment itself marks the residues involved in this feature with # symbols. The second row of the alignment is the consensus sequence of the CD record, whereas the third row contains the NP_004609 sequence, labeled “query”. At the bottom of the page, buttons allow Cn3D to be launched with various structural features highlighted. For example, if we are interested in nucleotide binding site II, Cn3D will launch with the view depicted in The
For an example query on finding and viewing conserved domains, see
To locate functional domains within a protein
To predict the function of a protein whose function is unknown
To establish evolutionary relationships across protein families
To interpret mutation studies
To predict the structure of a protein of unknown structure
Following the Domains link for any protein in Entrez, one can find the conserved domains within that protein. The
Information on the CDs contained within a protein can also be found from these databases and tools:
From any Entrez search: select the From the Structure Summary page of a MMDB record: this page displays the CDs within each protein chain immediately below the 3D Domain bar in the graphic display. Selecting the From an Entrez Domains search: choose From the CDD page: locate CDs by entering text terms into the search box and proceed as for an Entrez CD search. From a BLink report: select the From the BLAST main page: follow the RPS-BLAST link to load the CD-Search page.
Results from a CD search are displayed as colored bars underneath a sequence ruler. Moving the mouse over these bars reveals the identity of each domain; domains are also listed in a format similar to BLAST summary output (
These can be displayed by clicking a CD bar within a MMDB Structure Summary page or from a hyperlinked CD name on a CD-Search results page.
If members of a CD have MMDB records, one of these records can be viewed as a 3D image along with the sequence alignment using Cn3D (launched by selecting the pink dot on a CD-Search results page). As in other alignment views, colored capital letters indicate aligned residues, allowing the sequence of the protein sequence of interest to be mapped onto the available 3D structure.
For an example query on finding and viewing proteins with similar domain architectures, see
To locate related functional domains in other protein families
To gain insights into how a given CD is situated within a protein relative to other CDs
To explore functional links between different CDs
To predict the function of a protein whose function is unknown
To establish evolutionary relationships across protein families
Following the
From a CD-Search results page, click
From a CD-Summary page, click the
From an Entrez Domains searc, click the
These are described in At the
As illustrated in the sections above, there are numerous connections between the Structure resources and other databases and tools available at the NCBI. What follows is a listing of major tools that support connections.
Because Entrez is an integrated database system (
Although the BLAST service is designed to find matches based solely on sequence, the sequences of Structure records are included in the BLAST databases, and by selecting the PDB search database, BLAST searches only the protein sequences provided by MMDB records. A new
The BLink report represents a precomputed list of similar proteins for many proteins (see, for example, links from LocusLink records;
A particularly useful interface with the structural databases is provided on the
As stated elsewhere, all records in the MMDB are obtained originally from the Protein Data Bank (PDB) (
The CDD staff imports CD collections from both the Pfam and SMART databases. Links to the original records in these databases are located on the appropriate CD Summary page. Both Pfam and SMART are updated several times per year in roughly bimonthly intervals, and the CDD staff update CDD accordingly.
Structures displayed in Cn3D can be exported as a Portable Network Graphics (
Individual MMDB records can be saved/downloaded to a local computer directly from the Structure Summary page for that record.
Alignments of VAST neighbors can be saved/downloaded from the VAST Structure Neighbors page of any MMDB record. By selecting options in the
Users can download the NCBI Structure databases from the NCBI FTP site: mmdbdata: the current MMDB database (NOTE: these files can not be read directly by Cn3D). vastdata: the current set of VAST neighbor annotations to MMDB records nrtable: the current non-redundant PDB database pdbeast: table listing the taxonomic classification of MMDB records
CDD data can be downloaded from
The NCBI Taxonomy database is a curated set of names and classifications for all of the organisms that are represented in GenBank. When new sequences are submitted to GenBank, the submission is checked for new organism names, which are then classified and added to the Taxonomy database. As of April 2003, there were 176,890 total taxa represented.
There are two main tools for viewing the information in the Taxonomy database: the Taxonomy Browser, and Taxonomy Entrez. Both systems allow searching of the Taxonomy database for names, and both link to the relevant sequence data. However, the Taxonomy Browser provides a hierarchical view of the classification (the best display for most casual users interested in exploring our classification), whereas Entrez Taxonomy provides a uniform indexing, search, and retrieval engine with a common mechanism for linking between the Taxonomy and other relevant Entrez databases.
By the time the NCBI was created in 1988, the nucleotide
sequence databases (GenBank,
The Taxonomy Project started in 1991, in association with
the launch of Entrez (
To represent, manipulate, and store versions of each of the different database taxonomies, we wrote a stand-alone, tree-structured database manager, TaxMan. This also allowed us to merge the taxonomies into a single composite classification. The resulting hybrid was, at first, a bigger mess than any of the pieces had been, but it gave us a starting point that spanned all of the names in all of the sequence databases. For many years, we cleaned up and maintained the NCBI Taxonomy database with TaxMan.
After the initial unification and clean-up of the taxonomy for Entrez was complete, Mitch Sogin organized a workshop to give us advice on the clean-up and recommendations for the long-term maintenance of the taxonomy. This was held at the NCBI in 1993 and included: Mitch Sogin (protists), David Hillis (chordates), John Taylor (fungi), S.C. Jong (fungi), John Gunderson (protists), Russell Chapman (algae), Gary Olsen (bacteria), Michael Donoghue (plants), Ward Wheeler (invertebrates), Rodney Honeycutt (invertebrates), Jack Holt (bacteria), Eugene Koonin (viruses), Andrzej Elzanowski (PIR taxonomy), Lois Blaine (ATCC), and Scott Federhen (NCBI). Many of these attendees went on to serve as curators for different branches of the classification. In particular, David Hillis, John Taylor, and Gary Olsen put in long hours to help the project move along.
In 1995, as more demands were made on the Taxonomy database,
the system was moved to a SyBase relational database
(
In 1997, the
Organismal taxonomy is a powerful organizing principle in the
study of biological systems. Inheritance, homology by common descent,
and the conservation of sequence and structure in the determination
of function are all central ideas in biology that are directly related
to the evolutionary history of any group of organisms. Because of this,
taxonomy plays an important cross-linking role in many of the
The NCBI Taxonomy database is a curated set of names and classifications
for all of the organisms that are represented in
Of the several different ways to build a taxonomy, our group maintains
a If two organisms (A and B) are listed more closely together in the taxonomy
than either is to organism C, the assertion is that C diverged from the
lineage leading to A+B earlier in evolutionary history, and that A and B
share a common ancestor that is not in the direct line of evolutionary
descent to species C. For example, the current consensus it that the closest
living relatives of the birds are the crocodiles; therefore, our classification
does not include the familiar taxon Reptilia (turtles, lizards and snakes,
and crocodiles), which excludes the birds, and would break the phylogenetic
principle outlined above.
Our classification represents an assimilation of information from many
different sources (see
We do not rely on sequence data alone to build our classification, and we do not perform phylogenetic analysis ourselves as part of the taxonomy project. Most of the organisms in GenBank are represented by only a snippet of sequence; therefore, sequence information alone is not enough to build a robust phylogeny. The vast majority of species are not there at all, although about 50% of the birds and the mammals are represented. We therefore also rely on analyses from morphological studies; the challenge of modern systematics is to unify molecular and morphological data to elucidate the evolutionary history of life on earth.
Currently, more than 100 new species are added to the database daily, and the rate is accelerating as sequence analysis becomes an ever more common component of systematic research and the taxonomic description of new species.
The
The number and complexity of organisms in a submission can vary enormously. Many contain a single new name, others may include 100 species, all from the same familiar genus, whereas others may include 100 names (only half identified at the species level) from 100 genera (all of which are new to the Taxonomy database) without any other identifying information at all.
Some new organism names are found by software when the protein
sequence databases (
We often receive consults on submissions with explicitly new species
names that will be published as part of the description of a new species.
These sequence entries (like any other) may be designated “hold
until published” (
Occasionally, the same new genus name is proposed simultaneously for different taxa; in one case, two papers with conflicting new names had been submitted to the same journal, and both had gone through one round of review and revision without detection of the duplication. Although these duplications would have been discovered in time, the increasingly common practice of including some sequence analysis in the description of a new species can lead to earlier detection of these problems. In many cases, the new species name proposed in the submitted manuscript is changed during the editorial review process, and a different name appears in the publication. Submitters are encouraged to inform us when their new descriptions have been published, particularly if the proposed names have been changed.
We strongly encourage the submission of strain names for cultured bacteria, algae, and fungi and for sequences from laboratory animals in biochemical and genetic studies; of cultivar names for sequences from cultivated plants; and of specimen vouchers (something that definitively ties the sequence to its source) for sequences from phylogenetic studies. There are many other kinds of useful information that may be contained within the sequence submission, but these data are the bare minimum necessary to maintain a reliable link between an entry in the sequence database and the biological source material.
The
TaxBrowser is updated continuously. New species will appear on a daily basis as the new names appear in sequence entries indexed during the daily release cycle of the Entrez databases. New taxa in the classification appear in TaxBrowser on an ongoing basis, as sections of the taxonomy already linked to public sequence entries are revised.
The browser produces two different kinds of Web pages:
hierarchy pages, which present a familiar indented
flatfile view of the taxonomic classification,
centered on a particular taxon in the database; and taxon-specific pages, which summarize all of the information that
we associate with any particular taxonomic entry in the database. For
example, “hominidae” as a search term from the (
The taxon-specific browser display page shows all of the information
that is associated with a particular taxon in the Taxonomy database and
some information collected through links with related databases
( The name, taxid, rank,
There are two sets of links to Entrez records from the Taxonomy
Browser. The “subtree links” are accumulated up the tree
in a hierarchical fashion; for example, there are 16 million nucleotide
records and a half million protein records associated with the Chordata
(
“Direct links” will retrieve Entrez records that are
linked directly to this particular node in the taxonomy database. Many
of the Entrez domains (e.g., sequences and structures) are linked into
the taxonomy at or below the species level; it is a data error when a
sequence entry is directly linked into the taxonomy at a taxon somewhere
above the species level. For other Entrez domains (e.g., literature and
phylogenetic sets), this is not the case. A journal article may talk
about several different species but may also refer directly to the
The taxon-specific browser pages now also show the NCBI LinkOut links
to external resources. These include links to a broad range of different
kinds of resources and are provided for the convenience of our users; the
NCBI does not vouch for the content of these resources, although we do
make an effort to ensure that they are of good scientific quality. A
complete list of external resources can be found
There are several different ways to search for names in the Taxonomy database. If the search results in a terminal node in our taxonomy, the taxon-specific browser page is displayed; if the search returns with an internal (non-terminal) node, the hierarchical classification page is displayed.
Names can be duplicated in the Taxonomy database, but the taxonomy
browser can only be focused on a single taxon at any one time. If a
complete name search retrieves more than one entry from the taxonomy,
an intermediate name selection screen appears ( Searches for
There is a
The NCBI Taxonomy database is stored as a SyBase relational database, called TAXON. The NCBI taxonomy group maintains the database with a customized software tool, the Taxonomy Editor. Each entry in the database is a “taxon”, also referred to as a “node” in the database. The “root node” (taxid1) is at the top of the hierarchy. The path from the root node to any other particular taxon in the database is called its “lineage”; the collection of all of the nodes beneath any particular taxon is called its “subtree”. Each node in the database may be associated with several names, of several different nametypes. For indexing and retrieval purposes, the nametypes are essentially equivalent.
The Taxonomy database is populated with species names that have appeared in a sequence record from one of the nucleotide or protein databases. If a name has ever appeared in a sequence record at any time (even if it is not found in the current version of the record), we try to keep it in the Taxonomy database for tracking purposes (as a synonym, a misspelling, or other nametype), unless there are good reasons for removing it completely (for example, if it might cause a future submission to map to the wrong place in the taxonomy).
| File | Uncompresses to | Description |
|---|---|---|
| taxdump.tar.Z |
readme.txt | A terse description of the dmp files |
| nodes.dmp | Structure of the database; lists each taxid with its parent taxid, rank, and other values associated with each node (genetic codes, etc.) | |
| names.dmp | Lists all the names associated with each taxid | |
| delnodes.dmp | Deleted taxid list | |
| merged.dmp | Merged nodes file | |
| division.dmp | GenBank division files | |
| gencode.dmp | Genetic codes files | |
| gc.prt | Print version of genetic codes | |
| gi_taxid_nucl.dmp.gz | gi_taxid_nucl.dmp | A list of gi_taxid pairs for every live gi-identified sequence in the nucleotide sequence database |
| gi_taxid_prot.dmp.gz | gi_taxid_prot.dmp | A list of gi_taxid pairs for every live gi-identified sequence in the protein sequence database |
| gi_taxid_nucl_diff.dmp | gi_taxid_nucl_diff | List of differences between latest gi_taxid_nucl and previous listing |
| gi_taxid_prot_diff.dmp | gi_taxid_prot_diff | List of differences between latest gi_taxid_prot and previous listing |
For non-UNIX users, the file taxdmp.zip includes the same (zip compressed) data.
Each taxon in the database has a unique identifier, its taxid.
Taxids are assigned sequentially. When a taxon is deleted, its taxid
disappears and is not reassigned (
There are many possible types of names that can be associated
with an organism taxid in TAXON. To track and display the names
correctly, the various names associated with a taxid are tagged
with a nametype, for example “scientific name”,
“synonym”, or “common name”. Each taxid
When sequences are submitted to GenBank, usually only a scientific
name is included; most other names are added by NCBI taxonomists at
the time of submission or later, when further information is discovered.
For a complete description of each nametype used in TAXON, see
Scientific names, the only required nametype for a taxid, can be
further qualified into different classes. Not all “scientific
names” that accompany sequence submissions are true Linnaean
Latin binomial names; if the taxon is not identified to the species
level, it is not possible to assign a binomial name to it. For
indexing and retrieval purposes, TAXON needs to know whether the
scientific name is a Latin binomial name, or otherwise. A full
listing of the classes of TAXON scientific names can be viewed in
The treatment of duplicated names was discussed briefly in the section on the Taxonomy browser. For our purposes, there are four main classes of duplicated scientific names: (1) real duplicate names, (2) structural duplicates, (3) polyphyletic genera, and (4) other duplicate names.
There are several main codes of nomenclature for living organisms:
the Zoological Code (International Code of Zoological Nomenclature,
ICZN; for animals), the Botanical Code (International Code of Botanical
Nomenclature, ICBN; for plants), the Bacteriological Code (International
Code of Nomenclature of Bacteria, ICNB; for prokaryotes), and the Viral
Code (International Code of Virus Classification and Nomenclature, ICVCN;
for viruses). Within each code, names are required to be unique. When
duplicate names are discovered within a code, one of them is changed
(generally, the newer duplicate name). However, the codes are complex,
and not all names are subject to these restrictions. For example,
There is no real effort to make the scientific names of taxa unique among Codes, and among the relatively small set of names represented in the NCBI taxonomy database (20,000 genera), there are approximately 200 duplicate names (or about 1%), mostly at the genus level.
Early in 2002, the first duplicate species name was recorded in the
Taxonomy database.
In the Zoological and Bacteriological Codes, the subgenus that
Includes the type species is required to have the same name as the genus.
This is a systematic source of duplicate names. For these duplicates,
we use the associated rank in the unique name, e.g.,
Certain genera, especially among the asexual forms of Ascomycota
and Basidiomycota, are polyphyletic, i.e., they do not share a common
ancestor. Pending taxonomic revisions that will transfer species assigned
to “form” genera such as
We list many duplicate names in other nametypes (apart from our preferred “scientific name” for each taxon). Most of these are included for retrieval purposes, common names or the names of familiar paraphyletic taxa that we have not included in our classification, e.g. Osteichthyes, Coelenterata, and reptiles.
Aside from names, there are several optional types of information
that may be associated with a taxid. These are (1) rank, such as
species, genus or family; (2) genetic code, for translating proteins;
(3) GenBank division; (4) literature citations; and (5) abbreviated
lineage, for display in GenBank flat files. For more details on these
data types, see
The TAXON database is a node within the Entrez integrated retrieval
system (
Taxonomy was the first Entrez database to have an internal
hierarchical structure. Because Entrez deals with unordered sets
of objects in a given domain, an alternative way to represent these
hierarchical relationships in Entrez was required (see the section
The main focus of the Entrez Taxonomy
The default Entrez search is case insensitive and can be for any of
the names that can be found in the Taxonomy database. Thus, any of the
following search terms, Homo sapiens, homo sapiens, human, or Man, will
retrieve the node for
As for other Entrez databases, Taxonomy supports Boolean searching,
a
Each search result, listed in document summary (DocSum) format, may
have several links associated with it. For example, for the search
result Homo sapiens, the
A helpful list follows:
A search for Hominidae retrieves a single, hyperlinked entry.
Selecting the link shows the structure of the taxon. On the other
hand, a search for Hominidae[subtree] will retrieve a
nonhierarchical list of all of the taxa listed within the
Hominidae. A search for species[rank] yields a list of all species in
the Taxonomy database (108,020 in May 2002). Find the Taxonomy update frequency by selecting Entrez
An overview of the distribution of taxa in the DocSum list can be
seen if To filter out less interesting names from a DocSum list, add some
terms to the query, e.g., 2002/01/10[date] NOT uncultured[prop]
NOT unspecified[prop].
There are a variety of choices regarding how search results can be displayed in Taxonomy Entrez.
The Common Tree view shows an abbreviated view of the taxonomic
hierarchy and is designed to highlight the relationships between a
selected set of organisms. The ten species shown in
If there are more than few dozen taxa selected for the common
tree view, the display becomes visually complex and generally less
useful. When a large list of taxa is sent to the Common Tree display,
a summary screen is displayed first. For example, we currently list
727 families in the Viridiplantae (plants and green algae)
( The taxa are aggregated at the predetermined set of nodes in the
Taxonomy database that have been assigned “BLAST names”.
This serves an informal, very abbreviated, vernacular classification
that gives a convenient overview. The BLAST names will often not
provide complete coverage for all species at all levels in the tree.
Here, not all of our
There are several formatting options for saving the common tree display to a text file: text tree, phylip tree, and taxid list.
Hyperlinks to a common tree display can be made in two ways:
by specifying the common tree view in an Entrez query
by providing a list of taxids directly to the common tree
cgi function (for example,
The
As for any Entrez database, the contents are indexed by creating
term lists for each field of each database record (or taxid). For
There are five different index fields for names in Taxonomy Entrez.
The [lineage] and [subtree] index fields are a way to superimpose
the hierarchical relationships represented in the taxonomy on top of
the Entrez data model. For an example of how to use these field
limits for searching Taxonomy Entrez, see
The genetic code [gc], mitochondrial genetic code [mgc], and GenBank division [division] fields are all inherited within the taxonomy. The information in these fields refers to the genetic code used by a taxon or in which GenBank division it resides. Because whole families or branches may use the same code or reside in the same GenBank division, this property is usually indexed with a taxon high in the taxonomic tree, and the information is inherited by all those taxa below it. If there is no [gc] field associated with a taxon in the database, it is assumed that the standard genetic code is used. A genetic code may be referred to by either name or translation table number. For example, the two equivalent queries, standard[gc] and translation table 1[gc], each retrieves the set of organisms that use the standard genetic code for translating genomic sequences. Likewise, these two queries echinoderm mitochondrial[mgc] and translation table 9[mgc] will each retrieve the set of organisms that use the echinoderm mitochondrial genetic code for translating their mitochondrial sequences.
There are several useful terms and phrases indexed in the [prop] field. Possible search strategies that specify the prop field are explained below.
unspecified [prop] not identified at the species level
uncultured [prop] environmental sample sequences
unclassified [prop] listed in an “unclassified” bin
incertae sedis [prop] listed in an “incertae sedis” bin
We do not explicitly flag names as “unspecified” in
All of these search strategies below are valid. Taxonomy Eentrez displays only taxa that are linked to public sequence entries, and because sequence entries are supposed to correspond to the Taxonomy database at or below the species level, the Entrez query: terminal [prop] NOT “at or below species level” [prop] should only retrieve problem cases.
above genus level [prop]
above species level [prop]
“at or below species level” [prop] (needs explicit quotes)
below species level [prop]
terminal [prop]
non terminal [prop]
genetic code [prop]
mitochondrial genetic code [prop]
standard [prop] invertebrate mitochondrial [prop]
translation table 5 [prop]
The query “genetic code [prop]” retrieves all of the taxa at which one of the genomic genetic codes is explicitly set. The second query retrieves all of the taxa at which one of the mitochondrial genetic codes is explicitly set, and so on.
division [prop]
INV [prop]
invertebrates [prop]
The above terms index the assignments of the GenBank division
codes, which are divided along crude taxonomic categories (see
The remaining index fields are common to most or all Entrez domains, although some have special features in the taxonomy domain. For example, the field text word, [word], indexes words from the Taxonomy Entrez name indexes. Most punctuation is ignored, and the index is searched one word at a time; therefore, the search “homo sapiens[word]” will retrieve nothing.
Several useful terms are indexed in the properties field, [prop],
including functional nametypes and classifications, the rank level of
a taxon, and inherited values. See
More information on using the generic Entrez fields can be found
in the
Many of the Entrez databases (Nucleotide, Protein, Genome, etc.)
include an
To not retrieve such “exploded” terms, the unexploded
indexes should be used. This query will only retrieve the entries that
are linked directly to
Taxids are indexed with the prefix txid: txid9606 [orgn].
Source organism modifiers are indexed in the [properties] field, and such queries would be in the form: src strain[prop], src variety[prop], or src specimen voucher[prop]. These queries will retrieve all entries with a strain qualifier, a variety qualifier, or a specimen_voucher qualifier, respectively.
All of the organism source feature modifiers (/clone, /serovar, /variety, etc.) are indexed in the text word field, [text word]. For example, one could query GenBank for: “strain k-12” [text word]. Because strain information is inconsistent in the sequence databases (as in the literature), a better query would be: “strain k 12”[word] OR “strain k12”[word]. Note: explicit double-quotes may be necessary for some of these queries.
The Taxonomy
The checkboxes
Selecting one of the
A complete copy of the public NCBI taxonomy database is deposited
several times a day on our
Taxonomy BLAST reports (
The function library for the taxonomy application software in the
NCBI Toolkit is ncbitxc2.a (or libncbitxc2.a). The source code can be
found in the
In the early years of the project, Scott Federhen did all of the software and database development. In recent years, Vladimir Soussov and his group have been responsible for software and database development.
Scott Federhen (1990–present)
Andrzej Elzanowski (1994–1997)
Detlef Leipe (1994–present)
Mark Hershkovitz (1996–1997)
Carol Hotton (1997–present)
Mimi Harrington (1999–2000)
Ian Harrison (1999–2002)
Sean Turner (2000–present)
Rick Sternberg (2001–present)
If you have a comment or correction to our Taxonomy database,
perhaps a misspelling or classification or if something looks wrong,
please send a message to
Every node in the database is required to have exactly one
“scientific name”. Wherever possible, this is a
validly published name with respect to the relevant code of
nomenclature. Formal names that are subject to a code of
nomenclature and are associated with a validly published
description of the taxon will be Latinized uninomials above
the species level, binomials (e.g.
The scientific name is the one that will be used in all of
the sequence entries that map to this node in the Taxonomy database.
Entries that are submitted with any of the other names associated
with this node will be replaced with this name. When we change
the scientific name of a node in the Taxonomy database, the
corresponding entries in the sequence databases will be updated
to reflect the change. For example, we list
The “synonym” nametype is applied to both synonyms
in the formal nomenclatural sense (objective, nomenclatural,
homotypic
The “acronym” nametype is used primarily for the
viruses. The International Committee on Taxonomy of Viruses
(
The term “anamorph” is reserved for names applied
to asexual forms of fungi, which present some special nomenclatural
challenges. Many fungi are known to undergo both sexual and asexual
reproduction at different points in their life cycle (so-called
“perfect” fungi); for many others, however, only the
asexually reproducing (anamorphic or mitosporic) form is known (in
some, perhaps many, asexual species, the sexual cycle may have been
lost altogether). These anamorphs, often with simple and not especially
diagnostic morphology, were given Linnaean binomial names. A number of
named anamorphic species have subsequently been found to be associated
with sexual forms (teleomorphs) with a different name (for example,
The “misspelling” nametype is for simple misspellings. Some of these are included because the misspelling is present in the literature, but most of them are there because they were once found in a sequence entry (which has since been corrected). We keep them in the database for tracking purposes, because copies of the original sequence entry can still be retrieved. Misspellings are not listed on the TaxBrowser pages nor on the Taxonomy Entrez Info display views, but they are indexed in the Entrez search fields (so that searches and Entrez queries with the misspelling will find the appropriate node).
“Misnomer” is a rarely used nametype. It is used for names that might otherwise be listed as “misspellings” but which we want to appear on the browser and Entrez display pages.
The “common name” nametype is used for vernacular names associated with a particular taxon. These may be found at any level in the hierarchy; for example, “human”, “reptiles”, and “pale devil's-claw” are all used. Common names should be in lowercase letters, except where part of the name is derived from a proper noun, for example, “American butterfish” and “Robert's arboreal rice rat”.
The use of common names is inherently variable, regional, and often
inconsistent. There is generally no authoritative reference that
regulates the use of common names, and there is often not perfect
correspondence between common names and formally described scientific
taxa; therefore, there are some caveats to their use. For scientific
discourse, there is no substitute for formal scientific names.
Nevertheless, common names are invaluable for many indexing, retrieval,
and display purposes. The combination “
The “
The “in-part” nametype is included for retrieval
terms that have a broader range of application than the taxon or
taxa at which they appear. For example, we list reptiles and Reptilia
as in-part nametypes at our nodes
The “includes” nametype is the opposite of the in-part
nametype and is included for retrieval terms that have a narrower
scope of application than the taxon at which they appear. For example,
we could list “reptiles” as an “includes”
nametype for the
The “equivalent name” nametype is a catch-all category, used for names that we would like to associate with a particular node in the database (for indexing or tracking purposes) but which do not seem to fit well into any of the other existing nametypes.
The “genbank common name” was introduced to provide a mechanism by which, when there is more than one common name associated with a particular node in the taxonomy, one of them could be designated to be the common name that should be used by default in the GenBank flatfiles and other applications that are trying to find a common name to use for display (or other) purposes. This is not intended to confer any special status or blessing on this particular common name over any of the other common names that might be associated with the same node, and we have developed mechanisms to override this choice for a common name on a case-by-case basis if another name is more appropriate or desirable for a particular sequence entry. Each node may have at most one “genbank common name”.
There may be more than one acronym associated with a particular node in the Taxonomy database (particularly if several virus names have been synonymized in a single species). Just as with the “genbank common name”, the “genbank acronym” provides a mechanism to designate one of them to be the acronym that should be used for display (or other) purposes. Each node may have at most one “genbank acronym”.
The “genbank synonym” nametype is intended for those
special cases in which there is more than one name commonly used in
the literature for a particular species, and it is informative to have
both names displayed prominently in the corresponding sequence record.
Each node may have at most one “genbank synonym”. For example,
Although the use of either the anamorph or teleomorph name is
formally correct under the International Code of Botanical Nomenclature,
we prefer to give precedence to the telemorphic name as the
“scientific name” in the Taxonomy database, both to
emphasize their commonality and to avoid having two (or more) taxids
that effectively apply to the same organism. However, in many cases,
the anamorphic name is much more commonly used in the literature,
especially when sequences are normally derived from the asexual form
of the species. In these cases, the “genbank anamorph”
nametype can be used to annotate the corresponding sequence records
with both names. Each node may have at most one “genbank
anamorph”. For example:
Whenever possible, formal scientific names are used for taxa.
There are several codes of nomenclature that regulate the description
and use of names in different branches of the tree of life. These are:
the International Code of Zoological Nomenclature (
The viral code is less well developed than the others, but it includes
an official classification for the viruses as well as a list of approved
species names. Viral names are not Latin binomials (as required by the
other codes), although there are some instances (e.g.,
The zoological, botanical, and bacteriological codes mandate Latin
binomials for species names. They do not describe an official
classification (such as the
The fungi are subject to the botanical code. The cyanobacteria (blue-green algae) have been subject to both the botanical and the bacteriological codes, and the issue is still controversial.
“Authorities” appear at the end of the formal species
name and include at least the name or standard abbreviation of the
taxonomist who first described that name in the scientific literature.
Other information may appear in the authority as well, often the year
of description, and can become quite complicated if the taxon has been
transferred or amended by other taxonomists over the years. We do not
use authorities in our taxon names, although many are included in the
database listed as synonyms. We have made an exception to this rule in
the case of our first duplicated species name in the database,
All three of the codes of nomenclature for cellular organisms
provide for names at the subspecies level. The botanical and
bacteriological codes include the string “subsp.” in the
formal name; the zoological code does not, e.g.,
The botanical code (but none of the others) provides for two
additional formal ranks beneath the subspecies level, varietas and
forma. These names will include the strings “var.” and
“f.”, respectively, e.g.,
We list taxa with other subspecific names where it seems useful
and appropriate and where it is necessary to find places for names
in the sequence databases. For indexing purposes in the Genomes
division of Entrez, it is convenient to have strain-level nodes for
bacterial species with a complete genome sequence, particularly when
there are two or more complete genome sequences available for
different strains of the same species, e.g.,
Several other classes of subspecific groups do not have formal
standing in the nomenclature but represent well-characterized and
biologically meaningful groups, e.g., serovar, pathovar, forma
specialis, and others. In many cases, these may eventually be promoted
to a species; therefore, it is convenient to represent them
independently from the outset, e.g.,
Many other names below the species level have been added to the Taxonomy database to accommodate SWISS-PROT entries, where strain (and other) information is annotated with the organism name for some species.
In general, we try to avoid unqualified species names such as
When entries are not identified at the species level, multiple
sequences can be from the same unidentified species. Sequences
from multiple different unidentified species in the same genus
are also possible. To keep track of this, we add unique informal
names to the Taxonomy database, e.g., a meaningful identifier
from the submitters could be used. This could be a strain name,
a culture collection accession, a voucher specimen, an isolate
name or location—anything that could tie the entry to the
literature (or even to the lab notebook). If nothing else is
available, we may construct a unique name using a default formula
such as the submitter's initials and year of submission. This way,
if a formal name is ever determined or described for any of these
organisms, we can synonymize the informal name with the formal
one in the Taxonomy database, and the corresponding entries in
the sequence databases will be updated automatically. For example,
AJ302786 was originally submitted (in November 2000) as
Here are some examples of informal names in the Taxonomy database:
We use single quotes when it seems appropriate to group a phrase into a single lexical unit. Some of these names include abbreviations with special meanings.
“n. sp.” indicates that this is a new, undescribed species and not simply an unidentified species. “sp. nr.” indicates “species near”. In the example above, this indicates that this is similar to Camponotus gasseri. “aff.”, affinis, related to but not identical to the species given. “cf.”, confer; literally, “compare with” conveys resemblance to a given species but is not necessarily related to it. “s.l.”, sensu lato; literally, “in the broad sense”. “ex”, “from” or “out of” the biological host of the specimen.
Note that names with cf., aff., nr., and n. sp. are not unique and should have unique identifiers appended to the name.
Cultured bacterial strains and other specimens that have not been identified to the genus level are given informal names as well, e.g., Desulfurococcaceae str. SRI-465; crenarchaeote OlA-6.
Names such as Camponotus sp. 1 are avoided, because different
submitters might easily use the same name to refer to different species.
See
Sequences from environmental samples are given “uncultured” names. In these studies, nucleotide sequences are cloned directly from the environment and come from varied sources, such as Antarctic sea ice, activated sewer sludge, and dental plaque. Apart from the sequence itself, there is no way to identify the source organisms or to recover them for further studies. These studies are particularly important in bacterial systematics work, which shows that the vast majority of environmental bacteria are not closely related to laboratory cultured strains (as measured by 16S rRNA sequences). Many of the deepest-branching groups in our bacterial classification are defined only by anonymous sequences from these environmental samples studies, e.g., candidate division OP5, candidate division Termite group 1, candidate subdivision kps59rc, phosphorous removal reactor sludge group, and marine archael group 1.
These samples vary widely in length and in quality, from short
single-read sequences of a few hundred base pairs to high-quality,
full-length 16S sequences. We now give all of these samples anonymous
names, which may indicate the phylogenetic affiliation of the sequence,
as far is it may be determined, e.g., uncultured archaeon, uncultured
crenarchaeote, uncultured gamma proteobacerium, or uncultured enterobacterium.
See
Some groups of bacteria have never been cultured but can be characterized and reliably recovered from the environment by other means. These include endosymbiotic bacteria and organisms similar to the phytoplasmas, which can be identified by the plant diseases that they cause. We do not give these “uncultured” names, as above. These represent a special challenge for bacterial nomenclature, because a formal species description requires the designation of a cultured type strain. The bacteriological code has a special provision for names of this sort, Candidatus, e.g., Candidatus Endobugula or Candidatus Endobugula sertula; Candidatus Phlomobacter or Candidatus Phlomobacter fragariae. These often appear in the literature without the Candidatus prefix; therefore, we list the unqualified names as synonyms for retrieval purposes.
We allow informal names for unranked nodes above the species level as well. These should all be phylogenetically meaningful groups, e.g., the Fungi/Metazoa group, eudicotyledons, Erythrobasidium clade, RTA clade, and core jakobids. In addition, there are several other classes of nodes and names above the species level that explicitly do not represent phylogenetically meaningful groups. These are outlined below.
We are expected to add new species names to the database in a timely manner, preferably within a day or two. If we are able to find only a partial classification for a new taxon in the database, we place it as deeply as we can and list it in an explicit “unclassified” bin. As more information becomes available, these bins are emptied, and we give full classifications to the taxa listed there. In general, we suppress the names of the unclassified bins themselves so they do not appear in the abbreviated lineages that appear in the GenBank flatfiles, e.g., unclassified Salticidae, unclassified Bacteria, and unclassified Myxozoa.
If the best taxonomic opinion available is that the position of
a particular taxon is uncertain, then we will list it in an
“incertae sedis” bin. This is a more permanent assignment
than for taxa that are listed in unclassified bins, e.g.,
Fungi that were known only in the asexual (mitosporic, anamorphic)
state were placed formerly in a separate, highly polyphyletic category
of “imperfect” fungi, the Deuteromycota. Spurred especially
by the development of molecular phylogenetics, current mycological
practice is to classify anamorphic species as close to their sexual
relatives as available information will support. Mitosporic categories
can occur at any rank, e.g.,
The requirement that the Taxonomy database includes names from all of the entries in the sequence database introduces a number of names that are not typically treated in a taxonomic database. These are listed in the top-level group “Other”. Plasmids are typically annotated with their host organism, using the /plasmid source organism qualifier. Broad-host-range plasmids that are not associated with any single species are listed in their own bin. Plasmid and transposon names from very old sequence entries are listed in separate bins here as well. Plasmids that have been artificially engineered are listed in the “vectors” bin.
We do not require that Linnaean ranks be assigned to all of our taxa, but we do include a standard rank table that allows us to assign formal ranks where it seems appropriate. We do not require that sibling taxa all have the same rank, but we do not allow taxa of higher rank to be listed beneath taxa of lower rank. We allow unranked nodes to be placed at any point in our classification.
The one rank that we particularly care about is “species”. We try to ensure that all of the sequence entries map into the Taxonomy database at or below a species-level node.
The genetic codes and mitochondrial genetic codes that are
appropriate for translating protein sequences in different branches
of the tree of life are assigned at nodes in the Taxonomy database
and inherited by species at the terminal branches of the tree. Plastid
sequences are all translated with the standard genetic code, but many
of the mRNAs undergo extensive RNA editing, making it difficult or
impossible to translate sequences from the plastid genome directly.
The genetic codes are listed on our
GenBank taxonomic division assignments are made in the Taxonomy database and inherited by species at the terminal branches of the tree, just as with the genetic codes.
The Taxonomy database allows us to store comments and references
at any taxon. These may include hotlinks to abstracts in
Some branches of our taxonomy are many levels deep, e.g., the bony fish (as we moved to a phylogenetic classification) and the drosophilids (a model taxon for evolutionary studies). In many cases, the classification lines in the GenBank flatfiles became longer than the sequences themselves. This became a storage and update issue, and the classification lines themselves became less helpful as generally familiar taxa names became buried within less recognizable taxa.
To address this problem, the Taxonomy database allows us to flag taxa that should (or should not) appear in the abbreviated classification line in the GenBank flatfiles. The full lineages are indexed in Entrez and displayed in the Taxonomy Browser.
Sequence variations exist at defined positions within genomes and are responsible for individual phenotypic characteristics, including a person's propensity toward complex disorders such as heart disease and cancer. As tools for understanding human variation and molecular genetics, sequence variations can be used for gene mapping, definition of population structure, and performance of functional studies.
The Single Nucleotide Polymorphism database (dbSNP) is a public-domain archive for a broad collection of
simple genetic
The dbSNP accepts submissions for variations in any species and from any part of a genome. This document will provide you with options for finding SNPs in dbSNP, discuss dbSNP content and organization, and furnish instructions to help you create your own (local) copy of dbSNP.
The db The structure of the flanking sequence in dbSNP is a composite of bases
either assayed for variation or included from published sequence. We make the distinction to distinguish regions of sequence
that have been experimentally surveyed for variation (
In the physical mapping of nucleotide sequences, variations are used as positional markers. When mapped to
a unique location in a genome, variation markers work with the same logic as Sequence Tagged Sites
(
Variations that occur in functional regions of genes or in conserved non-coding regions can cause
significant changes in the complement of transcribed sequences, leading to changes in protein expression that can affect
aspects of the
The associations between variations and complex genetic traits are more ambiguous than simple, single-gene mutations that lead to a phenotypic change. When multiple genes are involved in a trait, then the identification of the genetic causes of the trait requires the identification of the chromosomal segment combinations, or haplotypes, that carry the putative gene variants.
The variations in dbSNP currently represent an uneven but large sampling of genome diversity. The human
data in dbSNP include submissions from the SNP Consortium, variations mined from genome sequence as part of the human genome
project, and individual lab contributions of variations in specific genes, mRNAs,
Systematic surveys of sequence variation will undoubtedly reveal sequences that are invariant in the sample. These observations can be submitted to dbSNP as NoVariation records that record the sequence, the population, and the sample size that were used in the survey.
The SNP database can be queried from the dbSNP homepage We organized the dbSNP homepage with links to documentation, FTP, and sub-query
pages on the
The dbSNP is now a part of the Entrez integrated information retrieval system (
Use this query module to select SNPs based on dbSNP record identifiers. These include reference SNP (refSNP) cluster ID numbers (rs#), submitted SNP Accession numbers (ss#), and local (or submitter) IDs for the same variations.
| Method class | Class code in Sybase, ASN.1, and XML |
|---|---|
| Denaturing high pressure liquid chromatography (DHPLC) | 1 |
| DNA hybridization | 2 |
| Computational analysis | 3 |
| Single-stranded conformational polymorphism (SSCP) | 5 |
| Other | 6 |
| Unknown | 7 |
| Restriction fragment
length polymorphism ( |
8 |
| Direct DNA sequencing | 9 |
| Population class | Description | Population class in Sybase, ASN.1, and XML |
|---|---|---|
| Central Asia | Samples from Russia and its satellite Republics and from nations bordering the Indian Ocean between East Asia and the Persian Gulf regions. | 8 |
| Central/South Africa | Samples from nations south of the Equator, Madagascar, and neighboring island nations. | 4 |
| Central/South America | Samples from Mainland Central and South America and island nations of the western Atlantic, Gulf of Mexico, and Eastern Pacific. | 10 |
| East Asia | Samples from eastern and south eastern Mainland Asia and from Northern Pacific island nations. | 6 |
| Europe | Samples from Europe north and west of Caucasus Mountains, Scandinavia, and Atlantic islands. | 5 |
| Multi-National | Samples that were designated to maximize measures of heterogeneity or sample human diversity in a global fashion. Examples include OEFNER|GLOBAL and the CEPH repository. | 1 |
| North America | All samples north of the Tropic of Cancer, including defined samples of United States Caucasians, African Americans, Hispanic Americans, and the NHGRI polymorphism discovery resource (NCBI|NIHPDR). | 9 |
| North/East Africa and Middle East | Samples collected from North Africa (including the Sahara desert), East Africa (south to the Equator), Levant, and the Persian Gulf. | 2 |
| Pacific | Samples from Australia, New Zealand, Central and Southern Pacific Islands, and Southeast Asian peninsular/island nations. | 7 |
| Unknown | Samples with unknown geographic provinces that are not global in nature. | 11 |
| West Africa | Sub-Saharan nations bordering the Atlantic north of the Congo River and central/southern Atlantic island nations. | 3 |
Use this module to construct a query that will select SNPs based on submission records by
laboratory (submitter), new data (called “new batches”, this query limitation is more recent than a
user-specified date), methods used to assay for variation (
Use sets of variation IDs collected from other queries to generate a variety of SNP reports.
The links in this section all point to the
Free Form is the most flexible query structure in dbSNP. Modeled on the NCBI Entrez retrieval system, queries can be conducted using multiple database field values to restrict a query to specific subsets of data. The Easy Form query is identical to the Free Form query, with the exception that the Easy Form query has a series of pull-down menus from which a value can be selected for the most popular query fields.
Use this query approach if you are interested in retrieving variations that have been mapped to a
specific region of the genome bounded by two STS markers. Other map-based queries are available through the NCBI
All links located on the left sidebar of the dbSNP homepage are also provided in text format at
the bottom of the page to support browsing by text-based Web browsers. Suggestions for improving database access by disabled
persons should be sent to:
The SNP database has two major classes of content: the first class is submitted data, i.e., original
observations of sequence variation ( The major sections of the report are described in the
A complete copy of the SNP database is publicly available and can be downloaded from the SNP
The essential component of a submission to dbSNP is the nucleotide sequence itself. dbSNP accepts
submissions as either genomic DNA or cDNA (i.e., sequenced mRNA transcript) sequence. Sequence submissions have a minimum
length requirement to maximize the specificity of the sequence in larger contexts, such as a reference genome sequence. We
also structure submissions so that the user can distinguish regions of sequence actually surveyed for variation from regions
of sequence that are cut and pasted from a published reference sequence to satisfy the minimum-length requirements.
| dbSNP variation
class |
Rules for assigning allele classes | Sample allele definition | Class code in Sybase,
ASN.1, and XML |
|---|---|---|---|
| Single Nucleotide
Polymorphisms (SNPs) |
Strictly defined as single base substitutions involving A, T, C, or G. | A/T | 1 |
| Deletion/Insertion
Polymorphisms (DIPs) |
Designated using the full sequence of the insertion as one allele, and either a fully defined string for the variant allele or a “-” character to specify the deleted allele. This class will be assigned to a variation if the variation alleles are of different lengths or if one of the alleles is deleted (“-”). | T/-/CCTA/G | 2 |
| Heterozygous
sequence |
The term heterozygous is used to specify a region detected by certain methods that do not resolve the polymorphism into a specific sequence motif. In these cases, a unique flanking sequence must be provided to define a sequence context for the variation. | (heterozygous) | 3 |
| Microsatellite or short
tandem repeat (STR) |
Alleles are designated by providing the repeat motif and the copy number for each allele. Expansion of the allele repeat motif designated in dbSNP into full-length sequence will be only an approximation of the true genomic sequence because many microsatellite markers are not fully sequenced and are resolved as size variants only. | (CAC)8/9/10/11 | 4 |
| Named variant |
Applies to
insertion/deletion polymorphisms of longer sequence features, such as retroposon dimorphism for |
(alu) / - | 5 |
| No-variation |
Reports may be submitted for segments of sequence that are assayed and determined to be invariant in the sample. | (NoVariation) | 6 |
| Mixed |
Mix of other classes | 7 | |
| Multi-Nucleotide
Polymorphism (MNP) |
Assigned to variations that are multi-base variations of a single, common length. | GGA/AGT | 8 |
Seven of the classes apply to both submissions of variations (submitted SNP assay, or ss#) and the non-redundant refSNP clusters (rs#'s) created in dbSNP.
The “Mixed” class is assigned to refSNP clusters that group submissions from different variation classes.
Class codes have a numeric representation in the
database itself and in the export versions of the data (
Alleles define variation class (
Each submitter defines the methods in their submission as either the techniques used to assay
variation or the techniques used to estimate allele frequencies. We group methods by method class (
Each submitter defines population samples either as the group used to initially identify
variations or as the group used to identify population-specific measures of allele frequencies. These populations may be one
and the same in some experimental designs. We assign populations a population class (
There are two sample-size fields in dbSNP. One field is called the
| Validation evidence | Description | Code in database for ss# | Code in FTP dumps for ss# | Code in database for rs# | Code in FTP dumps for rs# |
|---|---|---|---|---|---|
| Not validated | For ss#, no batch update or validation data, no frequency data (or frequency is 0 or 1). rs# status code is OR'd from the ss# codes. | 0 | Not present | 0 |
Not present |
| Multiple reporting | Status = 1 for an rs# with at least two ss# numbers; having at least one ss# is validated by a non-computational method. For a ss#, status = 1 if the method is non-computational. | 1 | 1 |
1,0 |
1 |
| With frequency | Frequency data is present with a value between 0 and 1. | 2 | 2 | 2 | 2 |
| Both frequency | For ss#, the method is non-computational and frequency data is present. If the ss# is a single cluster member, then the rs# code is set to 2. | 3 | 3 | 3/2 | 3 |
| Submitter validation | Submission of a batch update or validation section that reports a second validation method on the assay. | 4 | 4 | 4 | 4 |
If the rs# has a single ss# with code 1, then rs# is set to code 0.
For a single member rs where the ss# validation status = 1, the rs# validation status is set to 0.
Alleles typically exist at different frequencies in different populations; a very common allele in
one population may be quite rare in another population. Also, allelic variants can emerge as
Similar to alleles,
Some methods for detection of variation (e.g., denaturing high-pressure liquid chromatography or
DHPLC) can recognize when DNA fragments contain a variation without resolving the precise nature of the sequence change.
These data define an empirical measure of
dbSNP accepts individual genotypes for samples from publicly available repositories such as
dbSNP accepts individual assay records (ss# numbers) without validation evidence. When possible,
however, we try to distinguish high-quality validated data from unconfirmed (usually computational) variation reports. Assays
validated directly by the submitter through the
We release the content of dbSNP to the public in periodic “builds” that we synchronize with
the release of new genome assemblies ( The dbSNP build cycle starts with close of data for new submissions. We map all
data, including existing refSNP clusters and new submissions, to reference genome sequence if available for the organism.
Otherwise, we map them to non-redundant DNA sequences from GenBank. We then use map data on co-occurrence of hit locations to
either merge submissions into existing clusters or to create new clusters. We then annotate the new non-redundant refSNP (rs)
set on reference sequences and dump the contents of dbSNP in a variety of comprehensive and denormalized formats on the dbSNP
FTP site for release with the online build of the database.
Each build starts with a “close of data” that defines the set of new submissions that
will be mapped to genome sequence by
We annotate the non-redundant set of variations (refSNP cluster set) on reference genome sequence
Public release of a new build involves an update to the public database and the production of a
new set of files on the dbSNP FTP site. We make an announcement to the
Data submitted to dbSNP are clustered and provide a non-redundant set of variations for each
organism in the database. We maintain these clusters as refSNPs in dbSNP in parallel to the underlying submitted data. We
distinguish refSNPs from assay submissions by using an
refSNPs are compact sets of identifiers that are used to annotate variations on other NCBI
resources. A refSNP has a number of summary properties that are computed over all cluster members ( rs7412 has an average heterozygosity of 12.7% based on the frequency data
provided by the three submissions, and the cluster as a whole is validated because one of the underlying submissions has been
experimentally validated. rs7412 is annotated as a variation feature on RefSeq contigs, mRNAs, and proteins. Pointers in the
refSNP summary record direct the user to additional information on the three submitter Web sites, through the linkout
We compute summary measures for each refSNP to integrate data provided by each independent submitter.
Submitters can arbitrarily define variations on either strand of DNA sequence; therefore,
submissions in a refSNP cluster can be reported on the forward or reverse strand. The orientation of the refSNP and, hence,
its sequence and allele string, is set by a cluster exemplar. By convention, the clustering process We define clusters on shared locations (refSNPs) when we BLAST all
existing refSNPs against contig sequence. In cases where contig sequence is not available or the variation is defined in a
mRNA flanking sequence that will not map to a contig, we compute the refSNP set based on hits to the RefSeq or GenBank
sequences for the organism. We rank map hits as either LO or HI quality and parse the hits to assign a
Once the clustering process determines the orientation of all member sequences in a cluster, it will gather a comprehensive set of alleles for a refSNP cluster.
When the alleles of a submission appear to be different from the alleles of its parent refSNP, check the orientation of the submission for reverse orientation.
The best single measure of a variation's diversity in different populations is its average
Additional summary measures of variation include counts of populations and individuals sampled for this variation.
When reference genome assemblies are available, we use them as anchor sequence to place refSNP
clusters into a genomic context. We clean dbSNP flanking sequence with
We define a refSNP operationally as a variation at a location on an ideal reference chromosome. Such reference chromosomes are the goal of genome assemblies. However, work is still in progress in cases such as the human genome project; therefore, we must currently define a refSNP as a variation in the interim reference contig sequence. Every time there is a genomic assembly update, the interim reference contig sequence changes, and refSNPs must be updated or reclustered.
The reclustering process begins when NCBI updates the genomic assembly. We BLAST all existing refSNPs as well as any newly submitted SNPs (not yet bound to a refSNP cluster) against the genome assembly. Then, we cluster SNPs that co-locate at the same place on the genome into a single refSNP. Usually, new clusters are composed entirely of new submitted SNPs, or else the newly submitted SNPs cluster to an already existing refSNP. When newly submitted SNPs cluster among themselves, they are assigned to a new refSNP ID#, and when they cluster with an already existing refSNP, they are assigned to the cluster for that refSNP.
Sometimes a refSNP will co-locate with another existing refSNP. In this case, the refSNP with a higher ID number is retired, and all the submitted SNPs in its cluster are reassigned to the refSNP with the lower ID number.
Once the clusters are formed, the variation of a refSNP is the union of all possible alleles
defined in the set of submitted SNPs that composed the cluster.
We annotate
The two hits that define a weight 2 variation may not reflect
We do not believe that weight 3 and weight 10 variations have sufficient utility to warrant their annotation, but the mapping results for these variations are still available in dbSNP.
We annotate NoVariation records on NCBI RefSeq chromosomes, contig sequences, mRNAs, and proteins as a miscellaneous feature, or misc_feat. All dbSNP annotations also include a db_Xref cross-reference pointer back to dbSNP that uses the refSNP ID number.
GenBank records can be annotated only by their original authors. Therefore, when we find
high-quality hits of refSNP records to the
We annotate RefSeq mRNAs with variation features when the refSNP has a high-quality hit to the mRNA sequence. If the variation is in the coding region of the transcript and has a non-synonymous allele that changes the protein sequence, we also annotate the variation on the protein translation of the mRNA. The alleles in protein annotations are the amino acid translations of the affected codons.
The Map Viewer (
The summary values can be viewed or downloaded directly as a tab-delimited table if you
select the
| Functional class | Description | Database code |
|---|---|---|
| Locus region | Variation is within 2 Kb 5′ or 500 bp 3′ of a gene feature (on either strand), but the variation is not in the transcript for the gene. This class is indicated with an L in graphical summaries. | 1 |
| Coding | Variation is in the coding region of the gene. This class is assigned if the allele-specific class is unknown. This class is indicated with a C in graphical summaries. | 2 |
| Coding-synon | The variation allele is synonymous with the contig codon in a gene. An allele receives this class when substitution and translation of the allele into the codon makes no change to the amino acid specified by the reference sequence. A variation is a synonymous substitution if all alleles are classified as contig reference or coding-synon. This class is indicated with a C in graphical summaries. | 3 |
| Coding-nonsynon | The variation allele is nonsynonymous for the contig codon in a gene. An allele receives this class when substitution and translation of the allele into the codon changes the amino acid specified by the reference sequence. A variation is a nonsynonymous substitution if any alleles are classified as coding-nonsynon. This class is indicated with a C or N in graphical summaries. | 4 |
| mRNA-UTR | The variation is in the transcript of a gene but not in the coding region of the transcript. This class is indicated by a T in graphical summaries. | 5 |
| Intron | The variation is in the intron of a gene but not in the first two or last two bases of the intron. This class is indicated by an L in graphical summaries. | 6 |
| Splice-site | The variation is in the first two or last two bases of the intron. This class is indicated by a T in graphical summaries. | 7 |
| Contig-reference | The variation allele is identical to the contig nucleotide. Typically, one allele of a variation is the same as the reference genome. The letter used to indicate the variation is a C or N, depending on the state of the alternative allele for the variation. | 8 |
| Coding-exception | The variation is in the coding region of a gene, but the precise location cannot be resolved because of an error in the alignment of the exon. The class is indicated by a C in graphical summaries. | 9 |
Most gene features are defined by the location of the variation with respect to
transcript
We compute a functional context for sequence variations by inspecting the flanking sequence for gene features during the contig annotation process. We are also currently developing a method to do the same analysis on RefSeq/GenBank mRNAs.
Typically, one allele of a variation will be the same as the contig (contig reference), and the other allele will be either a synonymous change or a nonsynonymous change. In some cases, one allele will be a synonymous change, and the other allele will be a nonsynonymous change. If any allele is a nonsynonymous change, then the variation is classified as a nonsynonymous variation. Otherwise, the variation is classified as a synonymous variation.
The allele is the same as the contig (contig reference) and hence causes no change to the translated sequence.
The allele, when substituted for the reference sequence, yields a new codon that encodes the same amino acid. This is termed a synonymous substitution.
The allele, when substituted for the reference sequence, yields a new codon that encodes a different amino acid. This is termed a nonsynonymous substitution.
A problem with the annotated coding region feature prohibits conceptual translation. In this case, we note the variation class as coding, based solely on position.
Because functional classification is defined by positional and sequence parameters, two
facts emerge: (
When a SNP results in amino acid sequence change, knowing where that amino acid lies in
the protein structure is valuable. We provide this information using the following procedure. To find the location of a SNP
within a particular protein, we attempt to identify similar proteins whose structure is known by comparing the protein
sequence against proteins from the
The SNP database supports and encourages connections between assay records (ss#'s) and
supplementary data on the submitter's Web site. This connection is made using the
We make the following connections between refSNP clusters and other NCBI resources during the contig annotation process:
There are two methods by which we localize variations to known genes: (
When an original submitted SNP record shows a relationship between a SNP and a STS, we share the data with dbSTS and establish a link between the SNP and the STS record. We also examine refSNPs for proximity to STS features during contig annotation. When we determine that a variation needs to be placed within an STS feature, we note the relationship in the dbSNP table SnpInSts.
The contig annotation pipeline relates refSNPs to UniGene EST clusters based on shared
chromosomal location. We store Variation/
We connect individual submissions to PubMed record(s) of publications cited at the time of
submission. If you want to view links from PubMed to dbSNP, select
dbSNP stores the underlying variation data that define HLA alleles at the nucleotide level. The combinations of alleles that define specific HLA alleles are stored in dbMHC. dbSNP points to dbMHC at the haplotype level, and dbMHC points to dbSNP at both the haplotype and variation level.
dbSNP is a relational database with about 100 tables. NCBI deploys dbSNP in both MSSQL and Sybase
environments, and the public can download the full contents of the database from the dbSNP
A schema is a necessary part of constructing your own copy of dbSNP because it is a visual
representation of dbSNP and shows the logical relationship between data in dbSNP. It is available as a printable PDF
Data in dbSNP are organized into “zones” or boxes, depending on the nature of the
data. Each zone is color coded to allow the viewer to find the data more easily. The current color groupings are
The
The /schema subdirectory contains the files dbSNP_table.atx.gz and dbSNP_index.atx.gz. These files use standard SQL DDL language to create tables and indexes.
There are many utilities available to generate table/index creation statements
from a database. We use a tool called
For example, on UNIX operating systems, use gunzip to decompress the files: gunzip
dbSNP_table.atx.gz gunzip dbSNP_index.atx.gz to get the files dbSNP_table.atx and dbSNP_index.atx.
If you are using Sybase and the SQL server, use the command
Use “ftp -i” to turn off interactive prompting during multiple file transfers
to avoid having to hit “yes” to confirm transfer hundreds of times.
Because of the sheer volume of data, our tests to load data and then create indexes on a
Sun Sparc Sybase server using “fast bcp” took about 8 hours.
The file docsum.asn is the ASN structure definition file for both formats of the data. The 00readme file, located in the /specs subdirectory of the
The XML DTD comprises three files located in the /specs subdirectory of the
The FASTA report format provides the flanking sequence for each report of variation in dbSNP, as well as all submitted sequences that have no variation. ss FASTA contains all submitted SNP sequences in FASTA format, whereas rs FASTA contains all the reference SNP sequences in FASTA format. The FASTA data format is typically used for sequence comparisons using BLAST. BLAST SNP is useful for conducting a few sequence comparisons in the FASTA format, wherease multiple FASTA sequence comparisons will require the construction of a local BLAST database of FASTA formatted data and the installation of a local stand-alone version of BLAST.
The rs docsum flatfile report is generated from the ASN.1 datafiles and is provided in the
files “/ASN1_flat/ds_flat_chXX.flat”. As with all of the large report dumps, files are generated per chromosome (chXX in file name). Because flatfile reports are compact, they will not provide you with as much information as the ASN.1 binary report, but they are useful for scanning human SNP data manually because they provide detailed information at a glance. A full description of the information provided in the rs docsum flatfile format is available in the 00readme file, located in the SNP directory of the
The chromosome reports format provides an ordered list of RefSNPs in approximate chromosome coordinates. Chromosome reports is a small file to download but contains a great deal of information that might be helpful in identifying SNPs useful as markers on maps or contigs because the coordinate system used in this format is the same as that used for the NCBI genome Map Viewer. It should also be mentioned that the chromosome reports directory might contain the multi/ file and/or the noton/ files. These files are lists (in chromosome report format) of SNPs that hit multiple chromosomes in the genome and those that did not hit any chromosomes in the genome, respectively. A full description of the information provided in the chromosome reports format is available in the 00readme file, located in the SNP directory of the
A cycle of flank sequence masking and MegaBLAST alignment to the NCBI genome assembly of an organism is initiated either by the appearance of FASTA-formatted genome sequence for a new build of the assembly or by the significant accrual of newly submitted SNP data for that organism.
We prepare FASTA-formatted sequence for unclustered ss assays of a particular organism and run RepeatMasker (with the mixed case option set) on the sequence, using the appropriate ALU-repeat library. For human, we use
Both refSNP and subSNP FASTA sets are aligned to the genome assembly using MegaBLAST, and the SNP position is computed from the list of ungapped alignments returned with the
We also perform tailored BLAST algorithms to specialized subsets of the database that are known to fail or perform poorly in the standard pathway. Short sequences of 75 bases or fewer are BLASTed with word size = 22. To capture hits in cases where the variation length might break the alignment, each flank of microsatellites or indels are BLASTed independently and paired up in post processing to provide the SNP position. We perform high-stringency BLASTs for many of the heavily masked SNPs using wordsize = 40 and 99% quality in any alignment. This is an attempt to capture low-multiplicity hits of repetitive sequence without overburdening the BLAST algorithm with a large number of seeded alignments.
To reduce hit multiplicity in cases where SNPs align several times on the genome, we examine the relative hit quality of multiple hits. Hits are discarded when the mismatch count is greater than the minimum mismatch count. The mismatch count parameter is currently set to 3. We BLAST refSNPs and subSNPs against GenBank mRNA, RefSeq mRNA, and GenBank clone accessions by a similar procedure.
We send the output from the genome alignment (or clone alignment where a genome is not available) through a clustering procedure that assigns rs IDs to unclustered new submissions. In some cases, this clustering procedure will re-cluster existing refSNP clusters based on colocation on the genome. After clustering, we update BLAST output to synchronize the ss IDs and the re-clustered rs IDs to the associated refSNP cluster defined in the database. Location data are loaded into the database, wherease cumulative hit data for each mapped SNP are computed and loaded to the database. Finally, we update SNPFlankStatus.
When a protein is known to have a structure neighbor, dbSNP projects the RefSNPs located in that protein sequence onto sequence structures.
First, contig annotation results provide the SNP ID (snp_id), protein accession (protein_acc), contig and SNP amino acid residue (residue), as well as the amino acid position (aa_position) for a particular RefSNP. These data can be found in the dbSNP table, SNPContigLocusId. FASTA sequence is then obtained for each protein accession using the program idfetch, with the command line parameters set to:
We BLAST these sequences against the PDB database using blastall with the command line parameters set to:
Each SNP position in the protein sequence is used to determine its corresponding amino acid and amino acid position in the 3D structure from the BLAST result. These data are stored in the SNP3D table.
The Gene Expression Omnibus (
High-throughput hybridization array- and sequencing-based experiments have become increasingly common in molecular biology laboratories in recent years (
Because of the plethora of measuring techniques for molecular abundance in use, our primary goal in creating the Gene Expression Omnibus (
This chapter is both more current and more detailed than the previous literature report on GEO (
The entity–relationship diagram for GEO.
An actual example of three samples referencing one platform and contained in a single series.
| Accession prefix | Entity type | Subtype | Description |
|---|---|---|---|
| GPL | Platform | Commercial nucleotide array | Commercially available nucleotide hybridization array |
| Commercial tissue array | Commercially available tissue array | ||
| Commercial antibody array | Commercially available antibody array | ||
| Non-commercial nucleotide array | Nucleotide array that is not commercially available | ||
| Non-commercial tissue array | Tissue array that is not commercially available | ||
| Non-commercial antibody array | Antibody array that is not commercially available | ||
| GSM | Sample | Dual channel | Dual mRNA target sample hybridization |
| Single channel | Single mRNA target sample hybridization | ||
| Dual channel genomic | Dual DNA target sample hybridization, e.g., array CGH | ||
| SAGE | Serial analysis of gene expression | ||
| GSE | Series | Time–course | Time–course experiment, e.g., yeast cell cycle |
| Dose–response | Dose–response experiment, e.g., response to drug dosage | ||
| Other ordered | Ordered, but unspecified | ||
| Other | Unordered |
The three principle components (or entities) of GEO are modeled after the three organizational units common to high-throughput gene expression and array-based methodologies. These entities are called
The GEO repository is a relational database, which required that some fundamental implementation decisions were made:
(
(
Once a platform, sample, or series is defined by a submitter, an Accession number (i.e., a unique, stable identifier) is assigned (
Daily usage statistics evaluated over a 4-week period January 24 to February 20, 2002. Web server
It is very easy to use the Type in a valid public or private Select desired display options. Press the
Three types of display options are currently available:
To view one's own private, currently unreleased accessions, login with username and password at the bottom
A GEO Accession number is required to retrieve data from the GEO repository database (
Given a valid GEO Accession number, the Accession Display tool available on the GEO Web site provides a number of options for the retrieval and display of repository contents (see
Cumulative individual sample measurements submitted to GEO are shown. Data are presented by quarter since operations began on July 25, 2000.
Simple Omnibus Format in Text (SOFT) is a line-based, ASCII text format that allows for the representation of multiple GEO platforms, samples, and series in one file. In SOFT, metadata appear as label-value pairs and are associated with the tab-delimited text tables of platforms and samples. SOFT has been designed for easy manipulation by readily available line-scanning software and may be quite readily produced from, and imported into, spreadsheet, database, and analysis software. More information about SOFT and the submission process is available from the
There are several formats in which data can be deposited and retrieved from GEO. For deposit: (1) a file containing an ASCII-encoded text table of data can be uploaded, and metadata fields can be interactively entered through a series of Web forms; or (2) both data and metadata for one or more platforms, samples, or series can be uploaded directly in a format we call Simple Omnibus Format in Text, or SOFT (
Interactive and direct modes of communication are available for new data submissions and updating data submissions. The interactive Web form route is straightforward and most suited for occasional submissions of a relatively small number of samples. Bulk submissions of large data sets may be rapidly incorporated into GEO via direct deposit of SOFT formatted data.
Submissions may be held private for a maximum of 6 months; this policy allows data release concordant with manuscript publication. Such submissions are given a final Accession number at the time of submission, which may be quoted in a publication.
Currently, submissions are validated according to a limited set of criteria (see the
A quarterly, cumulative graph of the number of individual molecular abundance measurements in public submissions made through the first quarter of 2002 is shown in
| Field name | Description |
|---|---|
| Accession | GEO accession identifier |
| Author | Author of GEO sample |
| CloneID | Clone identifier of GEO sample's platform |
| Country | Country of GEO sample's submitter |
| email of submitter | |
| GBAcc | GenBank Accession of GEO sample's platform |
| Institute | Institute of GEO sample's submitter |
| Keyword | Keyword of GEO sample |
| ORF | Open reading frame (ORF) designation of GEO sample's platform |
| Organism | Organism of GEO sample and its parent taxonomic nodes |
| RefSeq | RefSeq accession of GEO sample's platform |
| SAGEtag | Serial analysis of gene expression (SAGE) 10-bp tag of GEO sample |
| Subtype | Subtype of GEO sample |
| Target ref | Target reference of GEO sample |
| Target src | Target source of GEO sample |
| Text Word | Word from description of GEO sample or sample's platform, and word from the titles of sample and its platform |
| Title | Titles of GEO sample and its platform |
The basic unit (defined by a unique identifier, or UID, in Entrez parlance) in Entrez ProbeSet is the GEO sample, fused with its affiliated platform and series information. The indexing process iterates through all platforms in the GEO database, extracting metadata and the data table and fishing for any sequence-based identifiers such as GenBank Accession, ORFs, Clone IDs, or SAGE tags. Each sample belonging to that platform is in turn assigned a new UID and indexed with the above platform information plus any related series metadata (
GenBank Accessions, PubMed references, and taxonomy information are also linked to the appropriate Entrez databases for cross-reference and appear in the
Extensive indexing and linking on the data in GEO are performed periodically and can be queried through Entrez ProbeSet (
Because samples are oftentimes organized into meaningful data sets within series, an example of retrieving a series and all the data of its associated samples and platform(s) is illustrative of the retrieval capabilities of the GEO Web site. For this example, to select a series of interest, we scan down a list of series in the GEO repository. However, to arrive at our series of interest, we could have just as well performed an Entrez ProbeSet query and followed GEO accession links to a sample and then to its related series, or followed links from PubMed to Entrez ProbeSet, and then to GEO. A step-by-step example of selecting a series of data and retrieving the data for this series from the GEO repository follows:
Select the linked number of public series from the table of Repository Contents given on the Scan down the list of The description of GSE27 on the In the A browser dialog states that it took 19 seconds to download the 5 MB SOFT file of data and metadata for one series (GSE27), seven samples (GSM992 to GSM1000), and one platform (GPL67).
Anticipated development of gene expression resources at NCBI is shown.
The GEO resource is under constant development and aims to improve its indexing, linking, searching, and display capabilities to allow vigorous data mining. Because the data sets stored within GEO are from heterogeneous techniques and sources, they are not necessarily comparable. For this reason, we have defined a ProbeSet to be a collection of GEO samples that contains comparable data. The selection of GEO samples into ProbeSets is necessary before integrating data in the GEO repository into other NCBI resources (see
How do I submit my data?
To submit data, an identity within the GEO resource must first be established. On first login, authentication and contact information must be provided. Authentication information (username and password) is used to identify users making submissions and updates to submissions. Contact information is displayed when repository contents are retrieved by others. This information is entered only once and can be updated at any time.
Is there a “hold until date” feature in GEO?
Yes. This feature allows a submitter to submit data to GEO and receive a GEO Accession number before the data become public. There is currently a 6-month limit to this hold period. All private data are publicly released eventually.
What kinds of data will GEO accept?
GEO was designed around the common features of most of the high-throughput gene expression and array-based measuring technologies in use today. These technologies include hybridization filter, spotted microarray, high-density oligonucleotide array, serial analysis of gene expression, and Comparative Genomic Hybridization (CGH) and protein (antibody) arrays but may be expanded in the future.
Does GEO archive raw data images?
No. However, a reference image will be optionally accepted (limited to 100 Kb in size in JPEG format). In combination with optional references to horizontal and vertical coordinates, this image can be used to provide the user of the data with a qualitative assessment of the data.
Are there any Quality Assurance (QA) measurements that are required by GEO?
Not at this time. These requirements may be added in the future.
How can I submit QA measurements to GEO?
QA measurements are currently optional. If QA measurements are performed at the image-analysis step, these can be submitted as additional sample data.
How can I make corrections to data that I have already submitted?
By logging in with a username and password, an option to update a previous submission or your contact information is given. Accession updates can also be made through a link from the Accession Display after logging in. Updating the data of an already existing and valid GEO Accession number will cause a new version of that data element to be created. Alterations of metadata will not create a new version. All of the various versions of a data element will remain in the database.
How are submitters authenticated?
In their first submission to GEO, submitters will be asked to select a username and password. This username and password can be used to submit additional data in the future without reentering contact information, as well as to authenticate the submitter when updating or resubmitting data elements under an existing GEO Accession number.
How do I get data from GEO?
You need not login to retrieve data. All the data are available for downloading. NCBI places no restrictions on the use of data whatsoever but does not guarantee that no restrictions exist from others. You should carefully read NCBI's data disclaimer, available on the GEO Web site.
What kind of queries and retrievals will be possible in GEO?
Currently, there are three ways to retrieve submissions. One way is by entering a valid GEO Accession number into the query box on the header bar of this page; this will take you to the Accession Display. Another is to use the platform, sample, and series lists, located on the GEO Statistics page. Sophisticated queries of GEO data and linking to other Entrez databases can be accomplished by using Entrez ProbeSet.
What does
GEO platforms (GPL prefix) may have related samples and, through those related samples, related series. GEO samples (GSM prefix) will always have one related platform and may have multiple, related series. GEO series (GSE prefix) will have at least one related sample and, through those related samples, will have at least one related platform. The
What is SOFT?
SOFT stands for Simple Omnibus Format in Text. SOFT is an ASCII text format that was designed to be a machine-readable representation of data retrieved from, or submitted to, GEO. SOFT output is obtained by using the Accession Display, and SOFT can be used to submit data to GEO. Please see Box 2 for more details.
What does the word “taxon” mean?
The NCBI's Taxonomy group has constructed and maintains a taxonomic hierarchy based upon the most recent information, which is described in
We gratefully acknowledge the work of Vladimir Soussov, as well as the entire NCBI Entrez team, especially Grisha Starchenko, Vladimir Sirotinin, Alexey Iskhakov, Anton Golikov, and Pramod Paranthaman. We thank Jim Ostell for guidance, Lou Staudt for discussions during our initial planning for GEO, and the extreme patience shown by Brian Oliver, Wolfgang Huber, and Gavin Sherlock when making the first data submissions. Admirable patience was also exhibited by Al Zhong during the development of the direct deposit validator. Special thanks go to Manish Inala and Wataru Fujibuchi for their continuing work on future features and tools.
| Source | Accessions | Description |
|---|---|---|
| NHGRI melanoma study | GSE1 | This series represents a group of cutaneous malignant melanomas and unrelated controls that were clustered based on correlation coefficients calculated through a comparison of gene expression profiles. |
| Stanford Microarray Database | GSE4 to GSE9, and GSE18 to GSE29 | These series represent microarray studies from the public collection of the Stanford Microarray Database ( |
| Cancer Genome Anatomy Project | GSE14 | This series represents the Cancer Genome Anatomy Project SAGE library collection. Libraries contained herein were either produced through CGAP funding or donated to CGAP. |
| Affymetrix Gene Chips™ | GPL71 to GPL101 | These platforms represent the latest probe attributes of the commercially available Affymetrix Gene Chips™ high density oligonucleotide arrays. |
| National Children's Medical Center Microarray Center | GSM1131 to GSM1345 | These samples represent direct deposits of data derived from Affymetrix Gene Chip™ arrays and come from the Microarray Center database at the National Children's Medical Center. |
Online Mendelian Inheritance in Man (
Trademark status. OMIMTM and Online Mendelian Inheritance in ManTM are trademarks of The Johns Hopkins University.
| MIM number range |
Explanation |
|---|---|
| 100000-199999 | Autosomal loci or phenotypes (entries created before May 15, 1994) |
| 200000-299999 | Autosomal loci or phenotypes (entries created before May 15, 1994) |
| 300000-399999 | X-linked loci or phenotypes |
| 400000-499999 | Y-linked loci or phenotypes |
| 500000-599999 | Mitochondrial loci or phenotypes |
| 600000- | Autosomal loci or phenotypes (entries created after May 15, 1994) |
MIM numbers are frequently preceded by a symbol. An asterisk (*) before a MIM number indicates that the entry describes a distinct gene or phenotype and that the mode of inheritance of the phenotype has been proved (in the judgment of the authors and editors) and that the phenotype described is not known to be determined by a gene represented by other asterisked entries in MIM.
A number sign (#) before a MIM number describing a phenotype indicates that the phenotype is caused by mutation in a gene represented by another entry and usually in any of two or more genes represented by other entries. The number sign is also used for phenotypes that result from specific chromosomal aberrations, such as Down syndrome, and for contiguous gene syndromes, such as Langer-Giedion syndrome. Whenever a number sign is used, the reason is stated at the outset of the entry.
The absence of an asterisk (or other sign) preceding the number indicates that the distinctness of the phenotype as a mendelizing entity or the characterization of the gene in the human is not established.
OMIM entries are authored and edited by
experts in the field and by the OMIM staff, based on information in
the published literature. All entries are assigned a unique, stable,
six-digit ID number and provide names and symbols used for the disorder and/or gene, a literature-based description, citations, contributor information, and creation and editing dates.
Because MIM is derived from the primary literature, the text is replete with citations and links to PubMed. OMIM authors create entries for each unique gene or Mendelian disorder for which sufficient information exists and do not wittingly create more
than one entry for each gene
MIM is organized into autosomal, X-linked,
Y-linked, and mitochondrial catalogs, and MIM numbers are assigned
sequentially within each catalog (
With the increasing complexity of biological information, OMIM makes a critical contribution by distilling what is known about a gene or disease into a single, searchable entry. The rich text of the OMIM entry, along with the source reference citations, make it easy to retrieve data of interest. The OMIM entry can then serve as a gateway to other sources of related information via the many curated and computed links within each entry.
The OMIM
The Synopsis of the human gene map has also been sorted alphabetically by disorder and is referred to as the Morbid Map.
OMIM can be found either by direct query
via
Queries can also be entered in the query
box shown on all Entrez database pages, selecting OMIM as the
database in the
The Gene Map can be accessed from an
individual OMIM entry via the cytogenetic location displayed when
appropriate under the entry titles. The Gene Map may also be queried
Title (default)
Details
Clinical Synopsis
Allelic Variants
mini-MIM
ASN.1
LinkOut
Related Entries
Genome Links
Nucleotide Links
Protein Links
PubMed Links
SNP Links
Structure Links
UniSTS Links
1. Enter query term or terms (example: renal failure hypertension).
2. Default display is
3. Select
4. Similarly, select
NOTE: In the same bar, the number
of entries to display and the format in which to display them can be
configured by use of the
OMIM is queried via a standard Entrez
query bar. The mechanics of selecting entries to display, how to
display them, and identifying related entries either within Entrez or
from external resources is also according to the Entrez/LinkOut
standard. The display options (
The OMIM
When viewing the text of an OMIM entry,
however, the navigational links serve as an electronic table of
contents. The section headings within the entry are listed similar to a table of contents, and selecting one moves the display to that section. Within an entry, selecting the
OMIM staff actively contributes to the curation of data in LocusLink. Thus, if the MIM number is represented in LocusLink, a reciprocal LocusLink link is provided in the section to the left. Other links provided by the LocusLink collaboration may also be listed in this section, e.g., links to Nomenclature, Reference Sequences, or UniGene clusters that are specific to the subject OMIM entry.
The sequence links in the LocusLink
section may be different from the Entrez indexing links available via
the
OMIM entries may also contain a link to
LinkOut for resources external to NCBI (see
At the top of any report page, or
associated with each entry in the query result page, are the links to
related data generated from Entrez (
Each OMIM entry has a unique number and is given a primary title and symbol. This is the title that is displayed in the document retrieval list. Alternative designations are listed below the primary title. Some entries contain information that is related but not synonymous to the primary topic and is not addressed in another entry (e.g., splice variants, phenotypic variants, etc.). This information is set off by the word “included” in the title. The first “included” title is displayed in the document retrieval list. The cytogenetic map location when known is given for each entry. When a disease shows genetic heterogeneity, more than one map location may be given. The “light bulb” icon at the end of text paragraphs links to related articles in PubMed. References within the text are linked to the complete citation at the end of the entry. There, the PubMed ID is linked to the PubMed abstract.
Some entries contain an
The OMIM data are available for
The OMIMTM database, including the
collective data contained therein, is the property of The Johns
Hopkins University, which holds the copyright thereto. The OMIM
database is made available to the general public subject to certain
restrictions. You may use the OMIM database and data obtained from
this site for your personal use, for educational or scholarly use, or
for research purposes only. The OMIM database may not be copied,
distributed, transmitted, duplicated, reduced, or altered in any way
for commercial purposes or for the purpose of redistribution without
a license from The Johns Hopkins University. Requests for information
regarding a license for commercial use or redistribution of the OMIM
database may be sent via email to
OMIM is funded by a contract from the National Library of Medicine and the National Human Genome Research Institute and by licensing fees paid to the Johns Hopkins University by commercial entities for adaptations of the database. The terms of these licenses are being managed by the Johns Hopkins University in accordance with its conflict of interest policies.
The
The online books are displayed one section at a time, with navigation provided to other parts of the current chapter or to other chapters within the book. Many of the books on the BookShelf can be browsed without any restriction at all; others have less flexibility for navigating the complete content. The publisher (or the owner of the content) defines the rules for access.
The books are linked to PubMed through research papers citations within the text. In the future, more links may be established between the BookShelf and other resources at NCBI, such as gene and protein sequences, genomes, and macromolecular structures.
The BookShelf provides a venue through which publishers and authors can make the full text of biomedical books available to the scientific community. As the BookShelf grows, we welcome proposals from authors, editors, or publishers of any “in-scope” texts, from undergraduate textbooks to more specialized publications, including collections of review articles or workshop proceedings. The scope of the BookShelf is broadly biomedical, including clinical works and those concerning basic biological and chemical sciences.
All books made available on the BookShelf have been provided in electronic format to
The complete contents of each book will be converted into
Any book that was printed from
Figures should be supplied in TIFF format, although GIF and JPEG formats may be accepted. The submitted text files are converted into XML according to the NCBI Book DTD; graphic files are converted into GIF and JPEG formats. Three hard copies of the book are also required, along with the electronic files.
The XML files are stored in a database. When a reader requests a book, chapter, or section, the XML is retrieved from the database and converted into HTML on the fly using Extensible Stylesheet Language Transformations (
There are three ways to access the content in BookShelf:
1. Through hyperlinked terms in PubMed abstracts
2. By a direct search using search terms or phrases (in the same way as the bibliographic database of PubMed is searched)
3. Through the Table of Contents of the book (note: some publishers restrict browsing through the entire book by disabling hyperlinks in the Table of Contents)
The BookShelf can be accessed from all PubMed abstract pages. When viewing a full PubMed abstract, select the This view was generated by selecting
A statistical weighting system based on the frequency of each phrase in a book section, relative to the rest of the book, is used to identify “good” phrases. A phrase that appears repeatedly in only a few sections and rarely in other parts of the book indicates a definitive phrase for those few sections; therefore, it ranks highly. Furthermore, the appearance of a phrase in the title, for example, has a greater value in the weighting system than one appearing solely in the text.
Each PubMed abstract can thus be linked to the appropriate book pages. This method allows two very dissimilar types of text—the dense, focused PubMed abstracts and the more descriptive book text—to find common ground.
| Field |
Use |
|---|---|
| [Author] | Search for the authors of books or chapters. |
| [Book] | Typically used with a Boolean expression to limit a search to a particular book. |
| [PmId] | Locate a journal article citation in a book by its PubMed ID. |
| [Rid] | Locate a particular book element (such as a figure or table) by its reference ID. |
| [Secondary Text] | Search for secondary text, e.g., units (mg/l, etc.) |
| [Title] | Search for words used in any title (book, chapter, section, subsection, figure, etc.). |
| [Type] | Locate a division of a book such as a section, chapter, or figure group. |
Filters are applied immediately following a search term, with no separating spaces, e.g., watson[author] AND cmed[book].
Book contents may be searched directly from the
Results are shown as a list of books in which the term is found, along with the number of sections, figures, and tables that contain the term ( If more than 20 sections, tables, and/or figures are found that contain the query term, a summary page, such as the one above, is displayed.
The document summary list is sorted with the most relevant documents shown at the top of the page. The sorting makes use of scores allocated to phrases as a measure of how relevant they are to a given section (a part of the statistical weighting system also used for linking PubMed abstracts to the books). For each book section found, the title, along with some context regarding the hierarchy of the section location (e.g., the chapter and book), is given. An icon is used to distinguish figure legends and tables from text sections ( The most relevant sections to the query appear at the top of the list. Note the icons that designate figure and table hits. The list may also be displayed in a brief format that lists only the section names that contain the term by choosing
Each HTML page of content seen in a Web browser represents one section of one chapter of a book, i.e., all of the content (including subsections and so on) within the first-level heading of a chapter. The amount of content this represents varies according to the structure of the original book. Some books have very long sections, some short, some a mixture; although on the whole, most chapters are divided into 3-10 sections.
The top of every page contains links to both short and detailed Tables of Contents and a description of the current location within the book ( Hyperlinks to various sections within a chapter appear within a navagation bar to the
All book content submitted to the BookShelf is converted to XML according to the public NCBI Book DTD.
Any files submitted in SGML or XML are converted to the Book DTD using
The XML for a book is generally a set of files. Each chapter and appendix is an independent file, as is the frontmatter. The book is pulled together by the book.xml file, which defines the structure of the book. For example, a book with two chapters and a bibliography at the end of the book would be structured as follows:
The book.xml file is composed of two parts. The first part defines all of the components that are required to build the book. These definitions occur within the <!DOCTYPE [ ] > tag. The second part builds the structure of the book. The root element is <book>. <book> contains whatever is in &frontmatter;, <body>, and <back>.
The book.xml example above refers to five external files: fm.xml, which contains all of the frontmatter of the book; ch1.xml, which contains chapter 1; ch2.xml, which contains chapter 2; biblist.xml, which contains the bibliography for the book; and graphics.xml, which defines the images. If any of these files is not valid according to the DTD or if the files are not found where they are defined in their <!ENTITY > declaration, then the book will not be valid.
All of the images, including figures, icons, and book-specific character graphics (see Special Characters below), are called out in the text as entities. The entities are defined in the graphics.xml file.
<!ENTITY ch2fu6 SYSTEM "data/mga/pictures/ch2/ch2fu6.gif" NDATA GIF>
<!ENTITY ch2fu7 SYSTEM "data/mga/pictures/ch2/ch2fu7.jpg" NDATA JPG>
<!ENTITY ch2fu8 SYSTEM "data/mga/pictures/ch2/ch2fu8.gif" NDATA GIF>
<!ENTITY ch2fu9 SYSTEM "data/mga/pictures/ch2/ch2fu9.gif" NDATA GIF>
<!ENTITY ch2fu10 SYSTEM "data/mga/pictures/ch2/ch2fu10.gif" NDATA GIF>
<!ENTITY ch2e1 SYSTEM "data/mga/pictures/ch2/ch2e1.gif" NDATA GIF>
<!ENTITY ch2e2 SYSTEM "data/mga/pictures/ch2/ch2e2.gif" NDATA GIF>
Graphic files are converted into GIF and JPEG formats and optimized for display on the Web. The images are not loaded into the database; they are retrieved from a file server when called by the HTML page.
Math expressions and chemical formulae and structures are handled as images.
The BookShelf uses the same character sets that PubMed Central uses (see
The XML files for each book are stored in a
The XML is converted to HTML using XSLT stylesheets. The look of the HTML pages is controlled further using CSS, which allow manipulation of colors, fonts, and typefaces (
The Table of Contents for a book is created from actual elements within the book content, rather than from the Table of Contents given in the book frontmatter of the hard copy. This ensures that the Table of Contents represents the content accurately as it is organized on BookShelf.
In the first version of the NCBI BookShelf project, Quark files were converted directly to HTML for display online. The result was effective, illustrating the value of having a textbook online and linked to PubMed; however, it was labor intensive, limiting, and not scaleable.
To simplify the delivery of books online and to allow for the expansion of linking within the Entrez system, NCBI decided to convert all content into a centralized XML format. The normalized XML content is easier to render, allows added value such as the addition of links to other NCBI databases, and simplifies the addition of new volumes.
PMC created a new DTD for the BookShelf project, which was based on the ISO-12803 DTD. As more books were converted to the NCBI Book DTD, changes had to be made to accommodate the data.
The NCBI Book DTD is a public DTD available on request from
How do I access the books at NCBI?
The online books can be accessed by direct searching in Entrez or through PubMed abstracts. After performing a general PubMed search, click on the author name of one of the search results to view the abstract. A hypertext link called
Which books are available at NCBI?
The book list is updated on a regular basis and can be viewed on the BookShelf homepage:
Can I search the books at NCBI?
Yes, the books can be searched either as a complete collection or as a single, selected book (restricted using search options found under
Can I browse the whole book?
The system has been designed so that the user is delivered to the most relevant book sections for a particular term or concept. Although navigation is possible in the immediate vicinity of the page to which you are delivered, it may not be possible to browse the complete book on BookShelf. The range of navigation for each book is determined on a case-by-case basis, in agreement with the publisher.
I am the publisher/author/editor of a book. How can I participate?
Please email
PubMed Central (PMC) is the National Library of Medicine's digital archive of full-text journal literature. Journals deposit material in PMC on a voluntary basis. Articles in PMC may be retrieved either by browsing a table of contents for a specific journal or by searching the database. Certain journals allow the full text of their articles to be viewed directly in PMC. These are always free, although there may be a time lag of a few weeks to a year or more between publication of a journal issue and when it is available in PMC. Other journals require that PMC direct users to the journal's own Web site to see the full text of an article. In this case, the material will always be available free to any user no more than 1 year after publication but will usually be available only to the journal's subscribers for the first 6 months to 1 year.
To increase the functionality of the database, a variety of links are added to the articles in PMC: between an article correction and the original article; from an article to other articles in PMC that cite it; from a citation in the references section to the corresponding abstract in PubMed and to its full text in PMC; and from an article to related records in other Entrez databases such as Reference Sequences, OMIM, and Books.
The
Every article citation in a table of contents includes one or more links (
In addition to the header information for the article itself, the upper part of a Full Text or PubLink page contains a variety of links, including links to other forms of the article, to related information in PubMed and other
An Abstract page is identical to a Full Text page that has been cut off at the end of the abstract.
A PMC PubLink page (
When an article has been cited by other articles in PMC, a “cited-in” link displays just under the article header information on both the Abstract and Full Text pages. Selecting this cited-in link gives you a list of the articles that have referenced the subject article (
PMC article citations may also be retrieved by doing a search in PMC or through a PubMed search. (In PubMed, use the
Participation by publishers in PMC is voluntary, although participating journals must meet certain editorial standards. A participating journal is expected to include all of its peer-reviewed primary research articles in PMC. Journals are encouraged to also deposit other content such as review articles, essays, and editorials. Review journals, and similar publications that have no primary research articles, are also invited to include their contents in PMC. However, primary research papers without peer review are not accepted.
Journals that deposit material in PMC may make the full text viewable directly in PMC or may require that PMC link to the journal site for viewing the complete article. In the latter case, the full text must be freely available at the journal site no more than 1 year after publication. In the case of full text that is viewable directly in PMC, which by definition is free, the journal may delay the release of its material for more than 1 year after publication, although all current journals have delays of 1 year or less.
In either case, the journal must provide
The rationale behind the insistence on free access is that continued use of the material, which is encouraged by open access, serves as the best test of the durability and utility of the archive as technology changes over time. PMC does not claim copyright on any material deposited in the archive. Copyright remains with the journal publisher or with individual authors, whichever is applicable.
Refer to
From Abstract and Full Text pages in PMC are links to related articles in PubMed and to related records in other Entrez databases, such as Nucleotides or Books. These are identical to the links between databases that you can find in any Entrez record.
PubMed Central is an XML-based publishing system for full-text journal articles. All journal content in the archive was either supplied in, or has been converted to, a Document Type Definition (
The content is displayed dynamically on the PMC site by journal, volume, and issue (if applicable). XML, Web graphics, PDFs, and supplemental data are stored in a
We receive journal content either directly from publishers or from publishers' vendors. This content includes:
SGML or XML of the articles to be deposited High-resolution images Supplemental data associated with the articles PDF versions of the articles
All of the text is converted to a central DTD, the PMC DTD, and the images are converted to Web format (GIF and JPEG). These files, along with any supplemental data or PDFs, are loaded into the database for linking, indexing, and retrieval (
Once all of the files in an issue or batch have been validated, they are converted to PMC XML (referred to as a PMC XML file or PXML) using XSLT (
For each publisher that submits SGML, an XML version of the DTD is created. This is used to parse the output of SX before the XML conversion is started. The XML version of the publisher's DTD is not used to validate the source data (because this has already been done using the original DTD).
XSLT requires that the input file be valid XML. If a DTD is not available for validation, the parser will check the syntax; it will also replace all of the character entities with the appropriate
After the XSLT conversion, the original character entities are converted to character entities that are valid under the PMC DTD. Character translation tables for each source DTD regulate this conversion.
The resulting XML is then validated against the PMC DTD.
Several other items are created along with the PXML file. These are:
1. An entity file (articlename.ent). This file lists all of the character entities (from the PMC DTD-defined entity sets) that are in the article. One entity file is created for each article. This information is loaded into the database and is used to prepare the final HTML file for display. A sample is below:
2. A PubMedID file (articlename.pmid). This file includes a set of reference citations in the format
When the article is converted, this information is collected from each journal citation in the bibliography and sent in a query to PubMed (using the
3. A source node file (articlename.src). When an article is passed through the XSLT conversion, a list is made of each node, or named piece of information, that is included in the file. As the conversion is running, each node that is being processed is recorded. When the conversion is complete, the processed node list is compared with the list of nodes in the source file, and any piece of information that was not processed is reported in a conversion log. A sample is below:
To accommodate the archiving requirements of the PubMed Central project, it is important that figures be submitted in the greatest resolution possible, in TIFF or EPS format. Figures in these formats will be available for data migration when formats change in the future, and PubMed Central will be able to keep all of the figures current.
For display on the PMC site, two copies of each figure are made: a GIF thumbnail (100 pixels wide) and a JPEG file that will be displayed with the figure caption when the figure is requested.
Supplemental data include any supporting information that accompanies the article but is not part of the article. They may be text files, Word document files, spreadsheet files, executables, video, and others.
Sometimes a journal has a Web site where all of this supplemental information is stored. In this case, PubMed Central establishes links from the article to the supplemental information on the publisher's site.
In other cases, the supplemental data files are submitted with the article to be loaded into the PMC database. Either way, the information concerning this supplemental data is collected in a Supplemental Data file, which includes the location of the supplemental file(s), the type of information that is available, and how the link should be built from the article. PMC does not validate any of the supplemental data files that are supplied.
Mathematical symbols and notations can be difficult to display in HTML because of built-up expressions and unusual characters. For the most part, expressions that are simple enough to display using HTML are not handled as math unless they are tagged as math specifically. Publishers that supply content to PMC handle math expressions in one of two ways: supplied images or encoding in SGML.
Any expression that cannot be tagged by the source DTD is supplied as an image. In this case, PMC will pass the image callout through to the PXML file and display the supplied image in the HTML file.
Several of the Source DTDs used by publishers to submit data to PMC are robust enough to allow coding of almost any mathematical expression in SGML. Most of these were derived from the Elsevier DTD; therefore, many of the elements are similar.
During article conversion, any items that are recognized as math are translated into Teχ. This would include any expression tagged specifically as a “formula” or “display formula,” as well as any free-standing expression that cannot be represented in HTML. These expressions include radicals, fractions, and anything with an overbar (other than accented characters). For example:
“ “1/2” would not be recognized as a math expression, but <fraction><numerator>1</numerator><denominator>2 </denominator></fraction> would be. “<radical>2x</radical>” would be recognized as a math expression, as would “<overbar>47X</overbar>.”
The SGML:
Converts to:
When the articles are loaded into the database, the equation markup is written into an Equation table. This table will also include the equation image, which will be created from the Teχ markup.
The image for the equation shown above in SGML and PXML (with Teχ) is:
Because all of the content in PubMed Central is in the same format—PMC DTD—loading articles into the database is relatively straightforward. Once an article is loaded into the production (public) database, it will retain its ArticleId (article ID number) in perpetuity. On loading, each article is validated against the PMC DTD. Also, any external files that are referenced by the XML are checked. If any file, such as a figure, is missing, the loading will be aborted.
The database loading software and daily maintenance programs perform several other tests to ensure the accuracy and vitality of the archive:
Journal identity. The Journal title being loaded is verified against the Duplicate articles. An article may not be loaded more than once. Any changes to the article must be submitted as a replacement article, which will use the same ArticleId. Publication date/delay. Rules for delay of publication embargo can be set up in the database to ensure that an issue will not be released to the public before a certain amount of time has passed since the publisher made the issue available. PubMed IDs. PubMedIDs for the article being loaded or any bibliographic citation in the article that are not defined in the PXML are looked up upon loading. Link updates. Links between related articles and from articles to external sources are updated daily.
The database has been designed to allow multiple versions of articles. In addition to article information, the database also stores information on content suppliers and publishers and journal-specific information.
PubMed Central uses a number of standard
Each publisher DTD defines a set of characters that may be used in their articles. Generally, these publisher DTDs use the same standard ISO character sets listed in
The supplied data also include groups of entities that are to be combined in the final document. Sometimes these are grouped in a tag such as:
and sometimes they are just positioned next to each other in the text. These combined entities must be mapped either to an ISO character or to a character in the PMC character set.
For the most flexibility in displaying characters across platforms, PMC uses UTF-8 encoding whenever possible. Because not all browsers support the same subset of UTF-8 characters and some characters cannot be represented in UTF-8, PMC displays characters as a combination of GIFs and UTF-8 characters, depending on the Browser/OS combination and the character to be displayed.
In the first version of the PMC project, the SGML and XML were loaded into a database in its native format. The HTML rendering software was then required to convert content from different sources into normalized HTML on the fly when a reader requested an article.
This was slow and cumbersome on the rendering side and was not scaleable. At that time, PMC was receiving content for about five journals in two DTDs, the keton.dtd from HighWire Press and the article.dtd from BioMed Central. The set-up for a new journal was difficult, and it soon became obvious that this solution would not scale easily.
To satisfy the archiving requirement for the PMC project and to simplify the delivery of articles online, PubMed Central decided to convert all content into a centralized format. The normalized content is easier to render, allows enhanced value such as links to other NCBI databases to be added, and simplifies content archiving.
PMC created a new DTD, which was strongly influenced by the BioMedCentral article.dtd and the keton.dtd. The original emphasis was on simplicity. As more and more articles from more and more journals were converted to the PMC DTD, changes had to be made to accommodate the data. The
Because the PMC DTD grew rapidly, it was feared that the original “simplicity” of its design would lead to confusing data structures. With more and more publishers inquiring about submitting content directly in the PMC DTD, PubMed Central decided that an independent review was necessary.
At approximately the same time, under the auspices of a Mellon Grant to explore ejournal archiving, Harvard University Library contracted with Can a common DTD be designed and developed into which publishers' proprietary SGML files can be transformed to meet the requirements of an archiving institution? If such a structure can be developed, what are the issues that will be encountered when transforming publishers' SGML files into the archive structure for deposit into the archive?
The requirement of the archival article DTD was defined as the ability to represent the intellectual content of journal articles. This
The NLM Archiving DTD will not be backwards-compatible with the pmc-1.dtd. It should be publicly available by the end of 2002, along with complete documentation for publishers and authors. A draft version is available (
Please refer to the
The RGB image demonstrates cytogenetic abnormalities in a cell from a secondary leukemia cell line.
Spectral Karyotyping (
The goal of the SKY/CGH database is to allow investigators to submit and analyze both clinical and research (e.g., cell lines) SKY and CGH data. The database is growing and currently has a total of about 700 datasets, some of which are being held private until published. Several hundred labs around the world use this technique, with many more looking at the data they generate. Submitters can enter data from their own cases in either of two formats, public or private; the public data is generally that which has already been published, whereas the private data can be viewed only by the submitters, who can transfer it to the public format at their discretion. The results are stored under the name of the submitter and are listed according to case number. The
Detailed information on how to submit data either to the SKY or CGH sectors of the database can be found through links on the
The submitter enters the written Chromosome images on the The written karyotype and a one-line case summary are also provided.
The CGH database displays gains, losses, and amplification of chromosomes and chromosome segments. The data are entered either by hand or automatically from various CGH software programs. In the manual format, the submitter enters the band information for each affected chromosome, describing the start band and stop band for each gain or loss, and the computer program displays the final karyotype with vertical bars to the left or right of each chromosome, indicating loss or gain, respectively (
Clinical data submitted include case identification, World Health Organization (WHO) disease classification code, diagnosis, organ, tumor type, and disease stage. To obtain the correct classification code, a link is provided to the NCI's
The references for the published cases are entered into the Case Information page and are linked to their abstracts in PubMed.
A colored karyotype with band overlay is presented to the submitter, who then builds each aberrant chromosome by cutting and pasting (by clicking with the mouse at appropriate breakpoints) ( Clicking on a chromosome brings up that chromosome with band overlay. Using the cursor, the operator cuts and pastes together each abnormal chromosome. The abnormal chromosome shown is a combination of chromosomes 10, 16, and 22.
To speed up the entry of cytogenetic data into the database, NCBI has built a computer program to automatically read short-form karyotypes, extract the information therein, and insert it into the SKY database ( The short-form written karyotype, entered in the karyotype field in (
Data submitters must use the same terminology for diagnosis (morphology) and organ site (topography) to permit comparison or combination of the data in the SKY/CGH database. From the many different disease classification systems, the
Quick Search can be found at the top of the SKY/CGH
The query results page displays information on all relevant cases, clones, and cells, along with details of SKY and/or CGH studies and clinical information for each case.
All of the public clinical and cytogenetic information can be searched. This format is currently under development.
This
All chromosomal bands, including breakpoints involved in chromosomal abnormalities, are linked to the Map Viewer database (
Links are provided to related Web sites including: chromosome databases (e.g., the Mitelman database); other NCI (e.g.,
One of the most intensely studied regions of the human genome is the Major Histocompatibility Complex (MHC), a group of genes that occupies approximately 4–6 megabases on the short arm of chromosome 6. The MHC genes, known in humans as Human Leukocyte Antigen (HLA) genes, are highly polymorphic and encode molecules involved in the immune response. The MHC database, Detailed single nucleotide polymorphism (SNP) mapping data of the HLA region KIR gene analysis data HLA diversity/anthropology data Multigene haplotype data HLA/disease association data Peptide binding prediction data
The MHC database is fully integrated with other NCBI resources, as well as with the International Histocompatibility Working Group (IHWG) Web site, and provides links to the IMmunoGeneTics HLA (IMGT/HLA) database.
This chapter provides a detailed description of current dbMHC resources and reviews dbMHC content and data computation protocols.
The most important known function of HLA genes is the presentation of processed peptides (antigens) to T cells, although there are many genes in the HLA region yet to be characterized, and work continues to locate and analyze new genes in and around the MHC (
The staff of the MHC database are currently accepting online submissions for the Primer/Probe/Mix component of dbMHC. Submissions can include typing data from any of the following: Sequence Specific Oligonucleotides (SSOs), Sequence Specific Primers (SSPs), SSO and/or SSP mixes, HLA typing kits, and Sequencing Based Typing (SBT). dbMHC allows submitters to edit their submissions online at any time.
A user can access dbMHC resources as a guest, that is, without having or being a member of an account. However, dbMHC guests will be unable to submit data to dbMHC or edit existing data. Data from a guest session will not be saved from session to session or even from frame to frame. Guests do have the option to download data from a particular session.
To create a dbMHC account, select the “Create an Account” from the left sidebar of the dbMHC homepage. Here, you will provide institutional information and specify an account administrator. Only the account administrator is allowed to do the following:
Enter new users. View or edit existing users. Change user permissions. This includes permission to modify allele reactivities and to enter new primers/probes/mixes or typing kits and to modify existing ones. Edit institutional information.
The
dbMHC can be used as a tool to help design primers/probes, to evaluate the reactivity patterns of potential probes, and to evaluate the polymorphism content of a particular gene.
Physical location Number and length of known alleles Allelic motifs Informativity Heterozygosity Primer sequences
Users will find the markers shown on the dbMHCms Web page useful because they provide evidence for genetic linkage and/or association of a genomic region with disease susceptibility. This resource was developed by NCBI in collaboration with A. Foissac, M. Salhi, and Anne Cambon-Thomsen, who provided the original data on MHC microsatellites in a series of updates (
The
Submitted sequences are aligned to reference locus sequences located in the dbMHC allele database based on a user-defined degree of nucleotide mismatch. After a brief analysis, the SBT interface will display exons, introns, and untranslated regions for each sequence. Allele assignments are listed according to matching order. Mismatched nucleotide positions are listed separately (in the lower frame of the SBT interface) after selecting
This Probe hybridization using Sequence Specific Oligonucleotides (SSOs). DNA amplification using Sequence Specific Primers (SSPs). Sequencing Based Typing (SBT) protocols. The Primer/Probe database can be used as a reference resource for probe design and is intended to provide necessary tools for the exchange of DNA typing data based on the above technologies. This database is composed of a Primer/ Probe Interface and a Typing Kit Interface. Each interface has a series of interactive frames that allow the user to access different database functions.
The Primer/Probe Interface provides information on individual reagents used for the typing of MHC or MHC-related loci. Users can access this information through the multiple functional frames within the interface. The words “primers” and “probes” represent SSOs, SSPs, SSP mixes, SSO mixes, and their combinations, which include nested PCR or SSO hybridization on group-specific amplifications. Nested PCR and group-specific amplification are both based on a primary amplification with an SSP mix.
The process of selecting a primer/probe begins by entering information in the upper frame of the Primer/Probe Interface page. Users enter search options for primers, probes, and mixes, which are grouped by type (SSO, SSP, SSO mix, and SSP mix), locus, and source (submitting institution). Primers/probes can also be selected by entering either the global or local name into the search field. Once this information is entered, the lower frame fills in with the Primer/Probe Listing. Primers/probes that are then selected by the user within the Primer/Probe Listing will be displayed in the large scroll box (in the upper frame) and can be downloaded in XML format.
Upon submission, dbMHC creates a unique global ID number for each primer/probe/mix/kit submitted. This global ID consists of three letters for the submitting institution and seven digits.
dbMHC uses the global ID number to store the entire history of a reagent. The global ID number will never change, but every time a user edits the sequence of a submitted primer/probe/mix/kit, that edit is given an incremental version number. Thus, all previous versions of a particular primer/probe/mix/kit are consequently accessible, and submitted primers/probes/mixes/kits can never be deleted.
A “local name” is the identifier given by submitters to their primers and probes.
This function is found in the lower frame of the Primer/Probe Interface page. Results of a Primer/Probe selection are displayed here and will include a unique global ID, local name, source, locus, and type for each primer/probe result. The Global Name (global ID) of each result provides a link to the Primer/Probe View or Mix View functions, and the check box associated with each primer/probe/mix can be used to generate a list of items in the Primer/Probe Interface for subsequent download in XML format.
The Primer/Probe View is located in the upper frame of the Primer/Probe Interface and appears as a result of selecting a global ID. This view will display the entire set of data of the SSP and/or SSO primers and/or probes that the user had selected in the Primer/Probe Interface. The data displayed include:
Local name Locus as specified by the submitter Global ID Date of last change (“last modified”) Probe orientation as specified by the submitter Corresponding allele sequence (Allele Seq:) Reagent type Optional filter, i.e., pre-amplification Version number Probe sequence (Probe Seq:) The annealing position of the 3' end of the probe as specified by the submitter Probe stringency
Primer/Probe View offers links that will enable a user to edit primer/probe information, change primer/probe stringency, and allow users to list alleles detected by a probe with a sequence alignment. If a list of alleles detected by a primer/probe without sequence alignment is wanted, Primer/Probe View can do this as well. Select
The primer/probe edit function allows users to enter new primers/probes into dbMHC or allows users to edit existing primer/probe data. All but the following three fields can be edited (these fields are set by dbMHC):
Global ID Version number Date of last change
Within the primer/probe edit frame, there is an alternative input field for primer/probe sequence called Allele Sequence. If primer or probe orientation is set to “reverse”, the Allele Sequence field will reverse complement the allele sequence as displayed in the alignment to generate the appropriate primer/probe sequence.
Before submitting primer/probe data, a user should define the matching stringency of the primer or probe in the Annealing Stringency field. Once the user has selected the matching stringency, the probe can be submitted. Submission of a primer/probe triggers dbMHC to begin an allele reactivity calculation that is based on the primer/probe sequence and the selected matching stringency. The result of this calculation is a list of alleles that might be detected by the submitted primer/probe. The user will find this detectable allele list in the Reactive Allele Listing (lower frame).
Reactivity scores characterize the reactivity between a primer/probe and an allele for primers and probes and are listed below:
Positive: probe anneals with allele. Weak: probe anneals sometimes with allele. Unclear: annealing cannot be predicted, no empirical information, allele has not been sequenced at the annealing position. Negative: probe does not anneal. dbMHC calculates primer/probe reactivity based on sequence similarity only and does not take into account laboratory experimental conditions such as magnesium concentration, temperature, etc.
A submitter's reactivity score can differ from the dbMHC's calculated reactivity score. If there is disparity between the submitter's score and the dbMHC score, users should regard the submitter's score as reliable. In cases of unexplainable disagreement, however, users should contact the submitting institution for further information.
The Reactive Allele Listing is located in the lower frame of the Primer/Probe Interface. It lists the alleles that might be detected by a selected primer/probe or mix (
A submitter can set individual allele reactivity scores either one-by-one or in a batch. The allele reactivity score for all alleles or for individual alleles that are unedited can be set the same as dbMHC's proposed allele reactivity score.
If the user chooses to set the reactivity score as a batch, be aware that alleles within the batch that are not currently displayed will be scored as well. If the user makes a mistake in the scoring,
Submitting allele reactivity scores will trigger a new dbMHC allele reactivity calculation. If the submitted primer/probe sequence is shorter than 10 nucleotides, dbMHC will use the score information of the alleles to extend the probe sequence.
Selecting
The alignment position of the 3' end of the probe is recorded for each allele. Both the sense and the reverse alignment positions are displayed if the alignment represents an SSP mix.
Only users with permission from an account administrator will be able to edit or add additional alleles to a Reactive Allele Listing for a particular primer/probe or mix. To edit the allele reactivity list, mark the “edit reactivity list” check box in the reactive allele alignment frame.
The Mix View function is located in the upper frame of the Primer/Probe Interface and can be accessed via the link on the global ID of the Primer/Probe Listing page.
It displays an entire set of mix data for selected SSO and SSP mixes:
Local name Mix type Locus as specified by the submitter Optional filter, i.e., pre-amplification Global ID Version number Date of last change List of probes as mix elements Mix stringency
The Mix View function contains links to the Mix Element function, where users can change or add elements of a mix. Users will also find links to the Reactive Allele Listing, where they may list alleles detected by a selected mix or selected individual elements of a mix that do not have a sequence alignment. Finally, users will find links to the reactive allele alignment function, where they can view alleles with sequence alignments that are detected by a selected mix or selected individual elements of a mix.
The Mix Edit function is located in the upper frame of the Primer/Probe Interface and allows users to enter new mixes or to edit existing mix data. Users can edit all but the following three fields (these fields are set by dbMHC):
Global ID Version number Date of last change
Annealing stringency defines the cumulative matching stringency of the elements of the mix. For SSP mixes, both the sense and the reverse primer must react with the allele with at least the defined stringency.
The Mix Element function is located in the lower frame of the Primer/Probe Interface and allows users to add elements to a mix or edit existing elements of a mix. To use the Mix Element function, the user must first specify the mix to be altered in the Mix Edit function and define a source for the primers/probes that the user wants listed. The mix element function displays only SSO probes for SSO mixes and displays SSP probes from SSP mixes in a sense column and a reverse orientation column. For mixes that contain probes from different sources, users must enter the mix elements separately.
All typing kits are identified by a global ID that is created by dbMHC, which will store the entire history of changes made to a typing kit. An incremental version number is given to every editing session of a typing kit; thus, even if some or all elements of a kit were deleted or altered by a user during an editing session, previous versions of the kit and kit elements will still be available.
The
Use the Typing Kit Interface to enter search parameters for typing kits. Typing kits are grouped by:
Type: SSO, SSP Locus Source: the submitting institution
Typing kits can also be selected by typing either the global or local name into the search field.
Search results, based on user-selected parameters, will be displayed in the Typing Kit Listing, which appears in the lower frame of the Typing Kit Interface. Typing kits selected from the Typing Kit Listing will be displayed in the large scroll box (of the upper frame). They either can be used for combined pattern interpretation of multiple kits or downloaded in XML format.
The Typing Kit Listing is displayed in the lower frame of the Typing Kit Interface. On the basis of the criteria selected, kits are listed with their unique global ID, local name, source, locus, and type. The global ID for each kit provides a link to the Typing Kit View (upper frame of the Typing Kit Interface). The check boxes associated with each typing kit are used to generate a list of items in the Kit Select view for subsequent interpretation of a pattern or for download in XML format.
If a primer/probe within an existing typing kit malfunctions and a user created a modification of the primer/probe, that modified primer/probe is considered by dbMHC to be an entirely new probe.
The kit that contains this modified primer/probe is considered by dbMHC as an entirely new kit. Therefore, when entering a sequence modification of primer/probe within a kit, a submitter must create a new kit by using
The Typing Kit View (upper frame of the Typing Kit Interface) lists the entire set of primer/probe data for selected SSO and SSP kits. Typing kit probe data include:
Local name Kit type Locus (with or without group-specific pre-amplifications) Global ID Version number Batch
If
Typing Kit Elements is displayed in the lower frame of the Typing Kit Interface and is used to display all of the elements (components) within a particular typing kit. This function will group the kit elements according to the kit locus groups and will display them as an ordered list. The global ID of each kit element serves as a link to either the Mix View or the Primer/Probe View of this element.
HLA typing kits are usually used to detect alleles in one or more loci. Within a typing kit, a particular locus and an optional pre-amplification define what is termed a “group”. Thus, one typing kit can contain several groups, with each group either consisting of the same locus and a different pre-amplification (or no pre-amplification) or consisting of different loci (with or without pre-amplification).
The Typing Kit Interface
The kit group function is located in the lower frame of the Typing Kit Interface and allows a user to add single elements to, or remove single elements from, a kit group. The kit group frame displays the elements for a particular locus selected by the user in the
Access to the kit edit function,
Access to the typing kit interpretation function,
The reactivity string or pattern entered will be interpreted as a heterozygote allele combination. Multiple kits can be combined. Users can set the degree of tolerance, which limits the number of false-positive or false-negative reactivities per locus. Each locus is analyzed separately. Allele assignments for each locus are listed according to the number of false-positive/false-negative reactivities.
Because of operating system limitations, the Alignment Viewer can only display up to 300 nucleotides in one line if the browser is Internet Explorer, whereas Netscape is not restricted by this limitation.
Several essential parts of dbMHC are based on Javascript interaction and dynamic text generation within a page. Users must be aware that many browsers are unable to properly interpret and display Javascript-generated text.
Netscape version 4.76 does not check for browser content size changes; therefore, users must manually resize to trigger the correct size recognition. Users may resize contents by using a “post” command, which will lead to a new download of the initial request instead of simply resizing the window.
Netscape 4.7 may sometimes cause fatal errors and does not allow users to copy and paste sequences from the alignment to the probe sequence field in the probe edit function.
Internet Explorer version 5.5 and Netscape version 6.2 will correctly interface with dbMHC.
Selecting
A “+” with a green background signals a detection of an allele by a kit element, a “w” with an orange background signals a weak detection, a “?” with a yellow background signals a lack of information, and an “r” with a red background signals a rejected interaction, although originally suggested by the prediction algorithm. If a kit is designed to detect only a subset of alleles, the display will be limited to this subset (see
| Table name | Column name | Data type | Column comment |
|---|---|---|---|
| allele | allele_nr | int | Unique identifier allele instance. |
| allele_id | int | Identifier for each allele. | |
| allele_name | varchar(30) | Full allele name defined by IHWG. | |
| allele_short | varchar(30) | Common use allele name. | |
| allele_group | int | Group associated with allele. | |
| compound_nr | int | Number identifying to which compound allele is a member. | |
| locus_id | int | Number identifying to which locus allele is a member. | |
| active | tinyint | Status labeling current active allele for this allele_id. | |
| db_version_nr | int | IHWG database version to which this allele belongs. | |
| user_id | int | Identifier or user who created/updated allele. | |
| user_date | datetime | Date when record was created/updated. | |
| block | block_nr | int | Unique identifier for block instance. |
| block_id | int | Identifier for each block. | |
| block_ord | int | Sequence order for each block. | |
| block_type | varchar(1) | Defines block type. | |
| locus_id | int | Identifier for locus for which block is a member. | |
| ref_pos | int | Start reference position. | |
| ref_length | int | Reference length. | |
| working_pos | int | Start working position. | |
| working_length | int | Working length. | |
| block_name | varchar(20) | Common name. | |
| active | tinyint | Status of row. | |
| db_version_nr | int | IHWG database version to which this block belongs. | |
| compound | locus_id | int | Locus identifier. |
| compound_nr | int | Compound number. | |
| job_trans | trans_nr | int | Unique identifier for transaction. |
| app_name | varchar(30) | Application name that processes transaction. | |
| app_arg | varchar(30) | Arguments to processing application. | |
| create_date | datetime | Time created. | |
| start_date | datetime | Time application started processing transaction. | |
| end_date | datetime | Time application completed processing transaction. | |
| status | int | Current status of transaction. | |
| priority | int | Defines priority level. | |
| message | varchar(255) | Any message that the processing application wants to record. | |
| kit | kit_nr | int | Unique identifier for kit instance. |
| kit_id | int | Kit identifier. | |
| kit_local_id | int | Submitter identifier. | |
| version_nr | int | Version number of kit. | |
| kit_name | varchar(30) | Submitter name. | |
| kit_global_id | varchar(30) | NCBI-defined name. | |
| kit_batch | varchar(30) | Submitter batch. | |
| type | int | Type. | |
| active | tinyint | Status labeling current active kit for this kit_id. | |
| source_id | int | Source identifier. | |
| user_id | int | User identifier. | |
| user_date | datetime | Date created/updated. | |
| kit_locus | kit_locus_nr | int | Unique identifier for kit locus instance. |
| kit_nr | int | ||
| kit_id | int | Kit identifier. | |
| locus_id | int | Locus identifier. | |
| locus_order_nr | int | Order for loci to be used. | |
| filter_id | int | Filter identifier. | |
| tube_id | int | Tube identifier. | |
| tube_order_nr | int | Order for tube to be used. | |
| active | tinyint | Status labeling current active kit locus for this kit id. | |
| user_id | int | User identifier. | |
| user_date | datetime | Date created/updated. | |
| locus | locus_id | int | Identifier for locus. |
| locus_NCBI_id | varchar(30) | NCBI locus identifier. | |
| locus_name | varchar(30) | Common name. | |
| display_id | int | Identifier of allele that should be used as the display reference. | |
| ref_id | int | Identifier of reference allele. | |
| locus_MIM_id | int | MIM locus identifier. | |
| locus_pub_id | int | PubMed locus identifier. | |
| display_order | int | Order number for display. | |
| mhc_snp | locus_id | int | Identifier for locus. |
| ref_pos | int | Reference position. | |
| ref_offset | int | Reference offset. | |
| subsnp_id | int | dbSNP's ss number. | |
| snp_length | int | SNP's length. | |
| pos_conv | compound_nr | int | Compound number. |
| working_pos | int | Working position. | |
| ref_pos | int | Reference position. | |
| ref_offset | int | Reference offset. | |
| blast_pos | int | BLAST position. | |
| probe_working | probe_working_nr | int | Unique identifier for each row. |
| tube_nr | int | Unique identifier for tube instance. | |
| probe_id | int | Tube (probe) identifier. | |
| user_id | int | User identifier. | |
| user_date | datetime | Date created/updated. | |
| active | tinyint | Active status. | |
| sequence | varchar(255) | Working sequence. | |
| sequence | seq_type | int | Sequence type. |
| seq_nr | int | Sequence number. | |
| seq_ord | int | Sequence order. | |
| sequence | varchar(255) | Character sequence. | |
| session | cur_session_nr | int | Current session number. |
| next_session_nr | int | Next avaliable session number. | |
| user_id | int | User identifier. | |
| create_date | datetime | Date created/updated. | |
| source | source_nr | int | Unique identifier source instance. |
| source_id | int | Source identifier. | |
| source_code | varchar(3) | Source code for use in naming tubes and kits. | |
| institution | varchar(50) | Source name. | |
| admin_id | int | User identifier for administrator for source. | |
| address | varchar(50) | Address. | |
| city | varchar(50) | City. | |
| state | varchar(50) | State. | |
| postal_code | varchar(20) | Postal code. | |
| country | varchar(50) | Country. | |
| phone | varchar(30) | Phone number. | |
| fax | varchar(30) | Fax number. | |
| varchar(50) | Email address. | ||
| active | tinyint | Status labeling current active source for this source id. | |
| user_id | int | User identifier. | |
| user_date | datetime | Date created/updated. | |
| options | int | Not used. | |
| info | varchar(255) | Comments/general information. | |
| tube | tube_nr | int | Unique identifier for tube instance. |
| tube_id | int | Tube identifier. | |
| tube_name | varchar(30) | Submitter name. | |
| tube_global_id | varchar(30) | NCBI-defined name. | |
| filter_id | int | Filter identifier. | |
| source_id | int | Source identifier. | |
| source_date | datetime | Not used. | |
| type | int | Type. | |
| sense | tinyint | Orientation. | |
| locus_id | int | Locus identifier. | |
| ref_pos | int | Submitted reference position. | |
| ref_offset | int | Submitted reference offset. | |
| stringency | int | Stringency set for annealing results. | |
| active | tinyint | Status labeling current active tube for this tube_id. | |
| user_id | int | User identifier. | |
| user_date | datetime | Date created/updated. | |
| recalc_date | datetime | Not used. | |
| db_version | int | IHWG database version to which this tube belongs. | |
| info | varchar(255) | Comments/general information. | |
| sequence | varchar(255) | Submitted sequence. | |
| source_nr | int | ||
| tube_allele | tube_allele_nr | int | Unique identifier for tube allele instance. |
| tube_id | int | Tube identifier. | |
| locus_id | int | Locus identifier. | |
| allele_id | int | Allele identifier. | |
| user_status | int | User status. | |
| system_status | int | System status. | |
| ref_pos | int | Reference position. | |
| ref_offset | int | Reference offset. | |
| block_nr | int | Not used. | |
| score | real | Annealing score. | |
| working_pos | int | Working position. | |
| for_tube_id | int | Forward tube identifier. | |
| for_working_pos | int | Forward working position. | |
| rev_tube_id | int | Reverse tube identifier. | |
| rev_working_pos | int | Reverse working position. | |
| sense | tinyint | Orientation. | |
| seq_known | int | Sequence known (sequence fully defined). | |
| active | tinyint | Status labeling current active tube allele for this tube_id allele_id combination. | |
| user_id | int | User identifier. | |
| user_date | datetime | Date created/updated. | |
| tube_probe | tube_probe_nr | int | Unique identifier for tube probe instance. |
| tube_nr | int | Unique identifier for tube instance. | |
| probe_id | int | Tube identifier for sub tube in mix. | |
| tube_id | int | Tube identifier. | |
| active | tinyint | Status labeling current active tube probe for this tube_id probe_id combination. | |
| users | user_id | int | User identifier. |
| user_date | datetime | Date created/updated. | |
| user_nr | int | Unique identifier for user instance. | |
| user_id | int | User identifier. | |
| first_name | varchar(30) | First name. | |
| last_name | varchar(30) | Last name. | |
| login | varchar(30) | Login. | |
| pwd | varchar(30) | Password. | |
| source_id | int | Source identifier. | |
| phone | varchar(30) | Phone number. | |
| fax | varchar(30) | Fax number. | |
| varchar(50) | Email address. | ||
| create_probe | tinyint | Permission to create probe. | |
| create_kit | tinyint | Permission to create kit. | |
| modify_probe | tinyint | Permission to modify probe. | |
| modify_react | tinyint | Permission to modify reactivity. | |
| modify_kit | tinyint | Permission to modify kit. | |
| info | varchar(255) | Comments/general information. | |
| active | tinyint | Status labeling current active user for this user_id. | |
| createdby_id | int | Created by identifier (another user_id). | |
| created_date | datetime | Date created/updated. | |
| source_admin | tinyint | Is user_id an administrator for source? |
The data submitted to dbMHC are stored in a Microsoft SQL (MSSQL) relational database.
The relationships between dbMHC database tables are depicted in
dbMHC's primer/probe database or interface is not curated. As such, the accuracy of the information presented in this database is dependent entirely upon the accuracy of the data submitted to dbMHC.
We suggest that each primer or probe submitted should be characterized by its complete sequence. In cases where submission of the complete sequence is impossible because the sequence information is considered proprietary, dbMHC offers the option to submit partial sequences. Submitted primer/probe specifications should comply with American Society of Histocompatibility and Immunogenetics (ASHI) (
The allele reactivity prediction tool for primers and probes available through dbMHC presents calculations based solely on sequence similarity and can therefore offer only suggestions for the possible allele reactivities of individual primers and probes. Users should not consider this tool to be 100% accurate. A reliable prediction of primer/probe allele reactivities requires complete sequence information, as well as reaction data that include annealing temperature and magnesium concentration. Because these data are not consistently available to dbMHC, we are unable to offer more than a suggestion for primer/probe allele reactivity.
dbMHC uses sequence comparisons to create a sequence interaction match or stringency grade that is compiled within a penalizing system. dbMHC's starting point for grading the interaction between a primer/probe and an allele is 100%, with each difference in sequence between the primer/probe and the allele causing a reduction in the remaining match grade by a certain percentage. The primary factors dbMHC uses to compute stringency grading include:
Nucleotide differences Nucleotide position and primer/probe type
| Probe | DNA template | |||
|---|---|---|---|---|
| A | T | G | C | |
| A |
|
|
|
|
| T |
|
|
|
|
| G |
|
|
|
|
| C |
|
|
|
|
Penalties calculated according to Peyret et al. (
dbMHC divides nucleotide interactions into five categories: perfect match, high match, medium match, poor match, and no match. dbMHC defines these categories by using the purine and pyrimidine interaction between the allele and the primer/probe, as well as the number of hydrogen bonds affected during the virtual pairing of the allele with the primer/probe. See
| 5' | 18 | 17 | 16 | 15 | 14 | 13 | 12 | 11 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 3' |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
| Primer | T | T | T | C | T | T | C | A | C |
|
T | C | C | G | T | G | T | C | |
| DNA template | T | T | T | C | T | T | C | A | C |
|
T | C | C | G | T | G | T | C |
An 18-mer primer anneals to a mismatched DNA. Primer and template are shown in the sense orientation. The substitution of C with A leads to the mismatch C-T with a penalty of 0.94 (refer to
This position has a 6% influence on the overall probe reactivity.
The overall probe score is 1 − (0.06 × 0.94) = 0.94.
|
|
10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
||
| Probe | T | T | T | C | T |
|
C | A | C | A | T | C | C | G | T | G | T | C | C | C | |
| DNA template | T | T | T | C | T |
|
C | A | C | A | T | C | C | G | T | G | T | C | C | C |
A 20-mer SSO anneals to a mismatched DNA. Both probe and template are shown in the sense orientation. The substitution of T with C leads to the probe-template mismatch T-G with a penalty of 0.42 (refer to
This position has a 70% influence on the overall probe reactivity.
The overall probe score is 1 − (0.7 × 0.42) = 0.71.
dbMHC uses a penalty system based on nucleotide position in its calculation of SSO/SSP interactions with different alleles. The system dbMHC uses indicates the extent to which an individual nucleotide position will affect the interaction stringency grade in a worst-case scenario, where guanines are in opposing sequence positions or cytosines are in opposing sequence positions. The maximum penalty given in the system is in the middle of an SSO and at the 3' end of an SSP, as shown in
If the initial interaction stringency grade for a particular primer/probe–allele interaction is 100% and a mismatch occurs in a position that carries a penalty of 100%, then dbMHC will reduce the interaction stringency grade to 0%. If the mismatch occurs in a position that carries a penalty of 95%, dbMHC will reduce the interaction stringency grade to 5%. A more detailed
dbMHC's reactive allele alignment algorithm searches for sequences with a match grade above the stringency level set by the submitter of each probe. Currently, the search algorithm searches within all loci that are part of dbMHC. Within each locus, the algorithm constructs compound sequences that represent the combined polymorphic positions of alleles of the same length. Alleles with insertions or deletions are handled separately. If a primer or probe matches with a certain position within the compound, all contributing alleles are checked for that position, whether or not they match.
The reactive allele alignment algorithm will check within a specified locus for sequences that have a match grade in accordance with primer/probe specifications at a user-indicated position. The algorithm then extends SSPs toward the 5' end of the probe, to a maximum of 10 nucleotides, and extends both sides of SSOs a maximum of 15 nucleotides, observing all polymorphic positions in matching alleles. The resulting probe extension is then used by the alignment algorithm to check for cross-reactivities within other loci.
If the submitted primer/probe sequence is shorter than 10 nucleotides and the user submits a score that accepts certain alleles and rejects others, the reactive allele alignment algorithm will use this information to refine the probe extension. If the probe submitter rejects all alleles containing a unique sequence motif in the vicinity of the probe sequence, the algorithm will generate the extended probe sequence such that it does not match the unique sequence motif.
The dbMHC provides a set of internal links to other NCBI resource homepages through either the black “quick link” bar located at the top of the dbMHC homepage or through the rotating HLA molecule located at the top left-hand corner of the dbMHC homepage. These links will take users to the following NCBI resources:
PubMed Nucleotide Protein OMIM BLAST Molecular Modeling database (MMdb)
The dbMHC Alignment Viewer provides locus-level linkage to NCBI's LocusLink and to dbSNP at the individual SNP level.
The Graphic View page is a hyperlink-enabled representation of the MHC region on chromosome 6. Locus-level links from the Graphic View page include:
dbSNP (at the SNP haplotype level) Map View LocusLink OMIM Nucleotide Protein PubMed Structure Books
The header and link column sections of dbMHC's Graphic View are the same as those on the main page. The content section of this page contains a selection box, called Choose Linked Resource, and three horizontally arranged sections, called Chromosome 6, HLA Class I, and HLA Class II. Genes listed in the HLA Class II section act as hyperlinks to the Web resource selected in Choose Linked Resource.
Currently, dbMHC provides links (from the left sidebar) to the following external sites: