chemfp.count_converters module
This module contains converters from count fingerprints to binary
The API is unlikely to change but may not be fully stable. The next release may have minor breaking changes, without the migration policy which applies to the stable portions of the chemfp API.
A CountConverter converts count fingerprints in one of
several different forms into a byte fingerprint. The
CountConverter.create_bytes() method, for example, starts with
a chemfp CountFingerprint.
There are also methods to parse an FPC-encoded fingerprint string and
an RDKit sparse fingerprint. By default theese work by converting the
input to a CountFingerprint then using create_bytes,
though some converters may have a specialized and faster interface.
The available convert converters are:
FoldedConverter_v1for index folding
BinaryConverter_v1for direct index mapping with no folding
RDKitCountSimConverter_v1for RDKit count simulation
SuperimposedConverter_v1for superimposed encoding
ScaledConverter_v1for scaled superimposed encoding
SeqConverter_v1for dense sequence encoding
ScaledSeqConverter_v1for dense scaled sequence encoding
These can be instanciated directly, or through the
get_count_converter() function, which takes a converter type
string and optional keywoard arguments. Processing error will raise a
CountConverterError subclass.
The underlying implementation and exception class structure is similar
to get_fingerprint_type().
For example:
>>> from chemfp import count_converters
>>> conv = count_converters.get_count_converter("folded", {"num_bits": 64})
>>> conv
FoldedConverter_v1(num_bits=64, hash=0)
>>> conv.get_type()
'folded/1 num_bits=64'
>>> conv.parse_fpc("1,70,127")
b'B
- class chemfp.count_converters.CountConverter
Bases:
objectBase class for converters from count fingerprint to byte fingerprints
Every converter has the attribute:
- num_bits: positive integer
The number of bits in the output fingerprint.
- combine_with_metadata(metadata: Metadata) tuple[Metadata, CountConverter]
Return a new metadata and a converter for the entire conversion pipeline.
For most processing, the new
Metadatais the result ofCountConverter.update_metadata()and the converter is left unchanged.In some cases the converter may need information from the metadata, most likely its num_bits, to describe the full conversion. In that case, this method will first create a new
CountConverterinstance with the needed details, then use the new count converter to create the new metadata, and return both new objects.This method may raise a ValueError if the combined metadata and converter do not have enough information or are incompatible.
- create_bytes(fp: CountFingerprint)
Convert a chemfp count fingerprint to a byte string
Calling the method on the base class will raise a NotImplementedError. It must be implemented in the subclass.
- Parameters:
fp (a
CountFingerprint) – a chemfp count fingerprint- Returns:
a binary fingerprint as bytes
- classmethod from_text_settings(settings: Dict[str, str] | None = None) CountConverter
Return a CountConverter instance based on the text settings
Calling the method on the base class will raise a NotImplementedError. It must be implemented in the subclass.
- classmethod get_kwargs_from_text_settings(settings: Dict[str, str] | None = None) Dict[str, Any]
- get_type(complete: bool = False) str
Return a canonical description of the converter as a string
Some parameters may not be included in the default type string, for example, if the value is not used or if a rarely used value equals the default. If complete is True then show all of the parameters.
The default implementation uses API hooks which are not yet part of the public API. If you derive your own subclass you will need to implement your own version.
- Parameters:
complete (bool) – If True, show all of the type parameters
- Returns:
a type string
- num_bits
- parse_fpc(s: str | bytes, stop_at_tab=True)
Parse an FPC-encoded count fingerprint to get a byte fingerprint
The input can be a text or byte string. By default parsing will stop at the end of string or the first tab. If stop_at_tab is False then the entire string must an FPC-encoded fingerprint.
This is equivalent to using
countops.parse_fpc()to parse the input FPC string to get aCountFingerprintthen converting that fingerprint to bytes withCountConverter.create_bytes(), which is how the default implementation works. A subclass may override this with a more optimized version.- Parameters:
s (str or bytes) – A string containing an FPC-encoded fingerprint
stop_at_tab (bool) – If True, stop parsing at the first tab
- Returns:
bytes
- parse_id_and_fpc(s: str | bytes, require_newline=False)
Parse a line from an FPC file to get the id and converted byte fingerprint
The input can be a text or byte string. By default the terminal newline may be present but is not required. If require_newline is True then the terminal newline must be present.
This is equivalent to using
countops.parse_id_and_fpc()to parse the input FPC string to get the id and aCountFingerprintthen converting that fingerprint to bytes withCountConverter.create_bytes(), which is how the default implementation works. A subclass may override this with a more optimized version.- Parameters:
s (str or bytes) – A string containing a fingerprint line from an FPC file.
require_newline (bool) – If True, the string must end with a newline.
- Returns:
a tuple of id string and bytes
- parse_rdkit_countfp(rdkit_countfp: _typing.RDKitCountFingerprint)
Parse an RDKit fingerprint binary string and convert to a byte fingerprint.
The rdkit_countfp may be a byte string generated by any of these RDKit’s DataStruct methods:
IntSparseIntVect().ToBinary()
UIntSparseIntVect().ToBinary()
LongSparseIntVect().ToBinary()
ULongSparseIntVect().ToBinary()
or any object with a ToBinary() method returning bytes in one of those formats.
The default implementation first converts the RDKit count fingerprint to a chemfp
CountFingerprint, then converts that to the bye string. A subclass may decide use a more optimized implementation.- Parameters:
rdkit_countfp (bytes, or its ToBinary() returns bytes) – An RDKit count fingerprint
- Returns:
bytes
- update_metadata(metadata: Metadata) Metadata
update the metadata to include details about this converter
The input
Metadatashould contains details about the input count fingerprint dataset. This method returns a new metadata instance which is a copy of the original metadata but with several fields changed so the result contains details about the output binary fingerprint dataset.- o The new ‘type’ combines the old ‘type’ and the converter’s
type string.
o The new ‘num_bits’ is the number of converter output bits.
o The new ‘num_bytes’ reflects the new ‘num_bits’
- exception chemfp.count_converters.CountConverterError
Bases:
ValueError,ChemFPErrorBase exception for many types of count converter errors
- get_help() str
Additional help with possible hints on how to fix the error.
Not currently used. This exists to match the API for get_fingerprint_type.
- get_reason() str
A one-line description of the reason for the error
- exception chemfp.count_converters.CountConverterTypeSyntaxError(type_str: str, reason: str)
Bases:
CountConverterErrorThe count converter type string does not follow the correct syntax.
- type_str: str
The invalid count converter type string
- reason: str
A short description of why the type string is invalid
- get_reason()
A one-line description of the reason for the error
- reason: str
- type_str: str
- exception chemfp.count_converters.CountConverterTypeUnknownError(base_name: str)
Bases:
CountConverterErrorThe requested count converter family is unknown.
The available attribute is:
- base_name: str
The convert family name, which is the first term in the type string, without version information.
- get_reason() str
A one-line description of the reason for the error
- class chemfp.count_converters.FoldedConverter_v1
Bases:
CountConverterConvert count fingerprint to binary using folding.
This converter implements modulo folding based on each feature index. The count is ignored. The basic method as pseudocode is:
output_fp = BinaryFingerprint(num_bits) for index, count in count_fingerprint: output_fp.SetBitOn(index % num_bits)
If hash is True then the hash of the index is used for the modulo folding:
output_fp = BinaryFingerprint(num_bits) for index, count in count_fingerprint: output_fp.SetBitOn(hash_function(index) % num_bits) return output_fp
Use hashing if the index values are not not well distributed after folding.
For example, suppose each index is a 15-bit atom triplet value where the first 5 bits come from the properties of atom 1, the next 5 bits come from the properties of atom 2, and the last 5 bits come from the properties of atom 3.
Now, the modulo operation is simply the remainder after division. To generate a 1024-bit fingerprint, the modulus is 1024, which means the bit positions with value 1024, 2048, 4096, etc. are all ignored (they are all multiple of 1024), leaving only the bit positions with values 1, 2, 4, … 512. These are the first 10 bits.
The result is a masking operation, where only the first 10 bits are used, and the remaining bits are ignored. In the above atom triplet, this means only the properties from atoms 1 and 2 will be used - atom 3 (which sets the 11th - 15th bits) will be ignored.
The hash method converts the input indices into new values which are not well-aligned with a fixed fingerprint length. This folded values incorporate more information from the input index, giving a better bit distribution.
- num_bits: positive integer
The number of bits in the output fingerprint.
- hash: bool
If True, use the index to seed a PRNG and generate the value too fold, instead of folding directly on the index.
- create_bytes(fp: CountFingerprint)
Fold a chemfp count fingerprint to a binary fingerprint
- Parameters:
fp (a
CountFingerprint) – a chemfp count fingerprint- Returns:
a binary fingerprint as bytes
- hash
- name = 'folded/1'
- parse_fpc(s: str | bytes, stop_at_tab=True)
Parse an FPC-encoded count fingerprint as folded bytes.
The input can be a text or byte string. By default parsing will stop at the end of string or the first tab. If stop_at_tab is False then the entire string must an FPC-encoded fingerprint.
The FPC-encoded count fingerprint is folded directly to byte fingerprint.
- Parameters:
s (str or bytes) – A string containing an FPC-encoded fingerprint
stop_at_tab (bool) – If True, stop parsing at the first tab
- Returns:
bytes
- parse_id_and_fpc(s: str | bytes, require_newline=False)
Parse a line from an FPC file as the id and folded byte fingerprint
The input can be a text or byte string. By default the terminal newline may be present but is not required. If require_newline is True then the terminal newline must be present.
The implementation parses the input FPC string directly to extract the id and create the folded fingerprint bytes.
- Parameters:
s (str or bytes) – A string containing a fingerprint line from an FPC file.
require_newline (bool) – If True, the string must end with a newline.
- Returns:
a tuple of id string and bytes
- parse_rdkit_countfp(rdkit_countfp: _typing.RDKitCountFingerprint)
Parse an RDKit fingerprint binary string and convert to a folded fingerprint.
The rdkit_countfp may be a byte string generated by any of these RDKit’s DataStruct methods:
IntSparseIntVect().ToBinary()
UIntSparseIntVect().ToBinary()
LongSparseIntVect().ToBinary()
ULongSparseIntVect().ToBinary()
or any object with a ToBinary() method returning bytes in one of those formats.
The RDKit count fingerprint is folded directly to a byte fingerprint.
- Parameters:
rdkit_countfp (bytes, or its ToBinary() returns bytes) – An RDKit count fingerprint
- Returns:
bytes
- class chemfp.count_converters.RDKitCountSimConverter_v1
Bases:
CountConverterConvert a count fingerprint to a binary fingerprint using RDKit count simulation.
RDKit count simulation is based around a list of countBounds of length w. The countBounds are used to turn a count into a set of w bits, where the first bit is set if the count is greater than or equal to the first countBound, and so on.
Generation is a two-step process. The first step uses modulo folding to convert the input count fingerprint to a reduced count fingerprint. The reduced count fingerprint has a length of
num_bits/wcounts, where the count for each position is the sum of the input counts for all input indices which fold to that position.The second step uses the countBounds to convert the total count at each reduced position to the corresponding bit pattern.
As pseudocode:
num_bins = len(countBounds) reduced_num_bits = num_bits // num_bins counts = [0] * reduced_num_bits for idx, count in count_fingerprint: counts[idx % reduced_num_bits] += count output_fp = BinaryFingerprint(num_bits) for i, count in enumerate(counts): if count == 0: continue for j, min_count in enumerate(countBounds): if count >= min_count: output_fp.SetOnBit(i * num_bins + j) return output_fp
The public attributes are:
- num_bits: positive integer
The number of bits in the output fingerprint.
- countBounds: a Python array.array of "I" (unsigned integers)
The countBounds used to set the bit pattern. The input countBounds are converted to a new Python array instance.
The default countBounds are [1, 2, 4, 8], which approximates a int(log2(n+1)) scaling for counts up to 15.
A countBounds of None means to use the default.
The countBounds in a type string is represented as a comma-separated list of values. The default generates the string “1,2,4,8”.
- countBounds
- create_bytes(fp: CountFingerprint)
Use RDKit count simulation to convert a chemfp count fingerprint to a binary fingerprint
- Parameters:
fp (a
CountFingerprint) – a chemfp count fingerprint- Returns:
a binary fingerprint as bytes
- name = 'rdkit-countsim/1'
- parse_fpc(s: str | bytes, stop_at_tab=True)
Parse an FPC-encoded count fingerprint as RDKit count simulation bytes.
The input can be a text or byte string. By default parsing will stop at the end of string or the first tab. If stop_at_tab is False then the entire string must an FPC-encoded fingerprint.
- Parameters:
s (str or bytes) – An FPC-encoded fingerprint
stop_at_tab (bool) – If True, stop parsing at the first tab
- Returns:
bytes
- parse_id_and_fpc(s: str | bytes, require_newline=False)
Parse a line from an FPC file as the id and RDKit count simulation byte fingerprint
The input can be a text or byte string. By default the terminal newline may be present but is not required. If require_newline is True then the terminal newline must be present.
The implementation parses the input FPC string directly to extract the id and create the RDKit count simulation fingerprint bytes.
- Parameters:
s (str or bytes) – A string containing a fingerprint line from an FPC file.
require_newline (bool) – If True, the string must end with a newline.
- Returns:
a tuple of id string and bytes
- class chemfp.count_converters.SuperimposedConverter_v1
Bases:
CountConverterConvert a count fingerprint to a binary fingerprint using random superimposed coding.
Each feature has an index and a count. For each feature, the index is used to seed a pseudo-random number generator to generates min(count, max_count)*bits_per_count random values from 0 to num_bits-1 (duplicates are allowed).
Using bits_per_count values > 1 appears to make the Tanimoto scores less reliable.
If the on-bit density of the output binary fingerprint is low (~5% or less) then the binary Tanimoto between two output fingerprints is a good approximation to the count Tanimoto between the two input count fingerprints.
- num_bits: positive integer
The number of bits in the output fingerprint.
- bits_per_count: positive integer
The number of bits to set for each count of each feature. The default is 1. It does not seem useful to set more than 1 bit per count.
- max_count: positive integer
If the feature count is larger than max_count then use max_count instead. The largest possible value is 2**32-1. The default is 1000, which prevent features like 0:1000000000 from generating 1 billion random values. NOTE: a future of this generator may place a hard upper-bound (likely 1000) on max_count.
- bits_per_count
- create_bytes(fp: CountFingerprint)
Superimpose a chemfp count fingerprint to a binary fingerprint
- Parameters:
fp (a
CountFingerprint) – a chemfp count fingerprint- Returns:
a binary fingerprint as bytes
- max_count
- name = 'superimposed/1'
- parse_fpc(s: str | bytes, stop_at_tab=True)
Parse an FPC-encoded count fingerprint as superimposed bytes.
The input can be a text or byte string. By default parsing will stop at the end of string or the first tab. If stop_at_tab is False then the entire string must an FPC-encoded fingerprint.
- Parameters:
s (str or bytes) – An FPC-encoded fingerprint
stop_at_tab (bool) – If True, stop parsing at the first tab
- Returns:
bytes
- parse_id_and_fpc(s: str | bytes, require_newline=False)
Parse a line from an FPC file as the id and superimposed byte fingerprint
The input can be a text or byte string. By default the terminal newline may be present but is not required. If require_newline is True then the terminal newline must be present.
The implementation parses the input FPC string directly to extract the id and create the superimposed fingerprint bytes.
- Parameters:
s (str or bytes) – A string containing a fingerprint line from an FPC file.
require_newline (bool) – If True, the string must end with a newline.
- Returns:
a tuple of id string and bytes
- parse_rdkit_countfp(rdkit_countfp: _typing.RDKitCountFingerprint)
Parse an RDKit fingerprint binary string to get the superimposed binary fingerprint
The rdkit_countfp may be a byte string generated by any of these RDKit’s DataStruct methods:
IntSparseIntVect().ToBinary()
UIntSparseIntVect().ToBinary()
LongSparseIntVect().ToBinary()
ULongSparseIntVect().ToBinary()
or any object with a ToBinary() method returning bytes in one of those formats.
The RDKit count fingerprint binary is superimposed directly to a byte fingerprint.
- Parameters:
rdkit_countfp (bytes, or its ToBinary() returns bytes) – An RDKit count fingerprint
- Returns:
bytes