API

Module contents

Access to all categorizations is provided directly at the module level, using the names of categorizations. To access the example categorization Excat, simply use climate_categories.Excat .

class climate_categories.Categorization(*, categories: dict[str, dict], name: str, title: str, comment: str, references: str, institution: str, last_update: date, version: None | str = None)

Bases: object

A single categorization system.

A categorization system comprises a set of categories, and their relationships as well as metadata describing the categorization system itself.

Use the categorization object like a dictionary, where codes can be translated to their meaning using cat[code] and all codes are available using cat.keys(). Metadata about the categorization is provided in attributes. If pandas is available, you can access a pandas.DataFrame with all category codes, and their meanings at cat.df.

name

The unique name/code

Type:

str

references

Citable reference(s)

Type:

str

title

A short, descriptive title for humans

Type:

str

comment

Notes and explanations for humans

Type:

str

institution

Where the categorization originates

Type:

str

last_update

The date of the last change

Type:

datetime.date

version

The version of the Categorization, if there are multiple versions

Type:

str, optional

hierarchical

True if descendants and ancestors are defined

Type:

bool

all_keys() → KeysView[str]

Iterate over all codes for all categories.

conversion_to(other: Categorization | str) → Conversion

Get conversion to other categorization.

If conversion rules for this conversion are not included, raises NotImplementedError.

property df: DataFrame

All category codes as a pandas dataframe.

extend(*, categories: None | dict[str, dict] = None, alternative_codes: None | dict[str, str] = None, name: str, title: None | str = None, comment: None | str = None, last_update: None | date = None) → CategorizationT

Extend the categorization with additional categories, yielding a new categorization.

Metadata: the name, title, comment, and last_update are updated automatically (see below), the institution and references are deleted and the values for version and hierarchical are kept. You can set more accurate metadata (for example, your institution) on the returned object if needed.

Parameters:

• categories (dict, optional) – Map of new category codes to their specification. The specification is a dictionary with the keys “title”, optionally “comment”, and optionally “alternative_codes”.

• alternative_codes (dict, optional) – Map of new alternative codes. A dictionary with the new alternative code as key and existing code as value.

• name (str) – The name of your extension. The returned Categorization will have a name of “{old_name}_{name}”, indicating that it is an extension of the underlying Categorization.

• title (str, optional) – A string to add to the original title. If not provided, “ + {name}” will be used.

• comment (str, optional) – A string to add to the original comment. If not provided, “ extended by {name}” will be used.

• last_update (datetime.date, optional) – The date of the last update to this extension. Today will be used if not provided.

Returns:

Extended categorization

Return type:

Categorization

static from_pickle(filepath: str | Path | IO[bytes]) → CategorizationT

De-serialize Categorization from a file written by to_pickle.

Note that this uses the pickle module, which executes arbitrary code in the provided file. Only load from pickle files that you trust.

static from_python(filepath: str | Path | IO[bytes]) → CategorizationT

De-serialize Categorization from a file written by to_python.

Note that this executes the python cache file. Only load from python cache files you trust.

classmethod from_spec(spec: dict[str, Any]) → CategorizationT

Create Categorization from a Dictionary specification.

classmethod from_yaml(filepath: str | Path | TextIO) → CategorizationT

Read Categorization from a StrictYaml file.

items() → ItemsView[str, Category]

Iterate over (primary code, category) pairs.

keys() → KeysView[str]

Iterate over the codes for all categories.

to_pickle(filepath: str | Path) → None

Serialize to a file using python’s pickle.

to_python(filepath: str | Path) → None

Write spec to a Python file.

to_spec() → dict[str, Any]

Turn this categorization into a specification dictionary ready to be written to a yaml file.

Returns:

spec – Specification dictionary understood by from_spec.

Return type:

dict

to_yaml(filepath: str | Path) → None

Write to a YAML file.

values() → ValuesView[Category]

Iterate over the categories.

class climate_categories.Category(codes: tuple[str, ...], categorization: Categorization, title: str, comment: None | str = None, info: None | dict = None)

Bases: object

A single category.

to_spec() → tuple[str, dict[str, str | dict | list]]

Turn this category into a specification ready to be written to a yaml file.

Returns:

(code – Primary code and specification dict

Return type:

str, spec: dict)

class climate_categories.Conversion(*, categorization_a: Categorization, categorization_b: Categorization, rules: list[ConversionRule], auxiliary_categorizations: list[Categorization] | None = None, comment: str | None = None, references: str | None = None, institution: str | None = None, last_update: date | None = None, version: str | None = None)

Bases: ConversionBase

Conversion between two categorizations.

This class collects functionality which needs access to the actual categorizations and categories.

categorization_a

The first categorization.

Type:

Categorization

categorization_b

The second categorization.

Type:

Categorization

auxiliary_categorizations

The auxiliary categorizations, if any.

Type:

list of Categorization, optional

rules

The actual rules for conversion between individual categories or sets of categories.

Type:

list of ConversionRule

comment

Notes and explanations for humans.

Type:

str, optional

references

Citable reference(s) for the conversion.

Type:

str, optional

institution

Where the conversion originates.

Type:

str, optional

last_update

The date of the last change.

Type:

datetime.date, optional

version

The version of the ConversionRules, if there are multiple versions.

Type:

str, optional

describe_detailed() → str

Detailed human-readable description of the conversion rules.

Sections are added for direct one-to-one mappings, one-to-many mappings, many-to-one mappings, and many-to-many mappings, respectively.

Factors are shown at the start of the line if they don’t equal 1, like this: -1 * IPCC1996 4 Agriculture to indicate that category 4 should be subtracted.

find_over_counting_problems() → list[OverCountingProblem]

Check if any category from one side is counted more than once on the other side.

Note that the algorithm at the moment can’t reliably detect all over counting problems and also some suspected problems might be fine under closer examination, so use this function only to generate hints for possible problems.

Returns:

problems – All detected suspected problems.

Return type:

list of OverCountingProblem objects

find_unmapped_categories() → tuple[set[Category], set[Category]]

Find categories for which no rule exists to map them.

Returns:

missing_categories_a, missing_categories_b – A list of categories missing from categorization_a and categorization_b, respectively.

Return type:

set, set

relevant_rules(categories: set[HierarchicalCategory], source_categorization: Categorization | None = None, simple_sums_only: bool = False) → list[ConversionRule]

Returns all rules which involve the given categories.

Parameters:

• categories (set of HierarchicalCategory) – The categories to limit the rules to.

• source_categorization (Categorization, optional) – The categorization that the categories are part of, either self.categorization_a or self.categorization_b.

• simple_sums_only (bool, default False) – If true, only consider rules where the given categories enter as simple summands (i.e. with a factor of 1).

Returns:

All rules which touch the given categories.

Return type:

relevant_rules

reversed() → Conversion

Returns the Conversion with categorization_a and categorization_b swapped.

class climate_categories.ConversionRule(factors_categories_a: dict[Category, int], factors_categories_b: dict[Category, int], auxiliary_categories: dict[Categorization, set[Category]], comment: str = '', csv_line_number: int | None = None, csv_original_text: str | None = None)

Bases: object

A rule to convert between categories from two different categorizations.

Supports one-to-one relationships, one-to-many relationships in both directions and many-to-many relationships. For each category, a factor is given which can also be negative to model relationships like A = B - C.

Using auxiliary_categories, a rule can be restricted to specific auxiliary categories only.

factors_categories_a

Map of categories from the first categorization to factors. For a simple addition, use factor 1, to subtract the category, use factor -1.

Type:

dict mapping categories to factors

factors_categories_b

Map of categories from the second categorization to factors. For a simple addition, use factor 1, to subtract the category, use factor -1.

Type:

dict mapping categories to factors

auxiliary_categories

Map of auxiliary categorizations to sets of auxiliary categories. Not all auxiliary categorizations need to be specified, and if an auxiliary categorization is not specified (or an empty set of category codes is given), the validity of the rule is not restricted. If an auxiliary categorization is specified and categories are given, the rule is only valid for the given categories. If multiple auxiliary categorizations are given, the rule is only valid if all auxiliary categorizations match.

Type:

dict[Categorization, set[Category]]

comment

A human-readable comment explaining the rule or adding additional information.

Type:

str

cardinality_a

The cardinality of the rule on side a. Is “one” if there is exactly one category in factors_categories_a, and “many” otherwise.

Type:

str

cardinality_b

The cardinality of the rule on side b. Is “one” if there is exactly one category in factors_categories_b, and “many” otherwise.

Type:

str

is_restricted

The rule is restricted if and only if for at least one auxiliary categorization at least one category is specified, so that the rule is only valid for a subset of cases. Otherwise, the rule is unrestricted and valid for all cases.

Type:

bool

format_human_readable(categorization_separator: str = '⮁\n') → str

Format the rule for humans.

Parameters:

categorization_separator (str, optional) – The categorization_separator is printed between the categories from the source categorization and the categories from the target categorization to make the difference clear.

Returns:

human_readable – The rule in a format optimized for error-free parsing by humans.

Return type:

str

format_with_lineno() → str

Human-readable string representation of the rule with information in which line in the CSV file it was defined, if that is available.

reversed() → ConversionRule

Return the ConversionRule with categorization_a and categorization_b swapped.

to_spec() → ConversionRuleSpec

Return a serializable specification.

Returns:

spec

Return type:

ConversionRuleSpec

class climate_categories.HierarchicalCategorization(*, categories: dict[str, dict], name: str, title: str, comment: str, references: str, institution: str, last_update: date, version: None | str = None, total_sum: bool, canonical_top_level_category: None | str = None)

Bases: Categorization

In a hierarchical categorization, descendants and ancestors (parents and children) are defined for each category.

total_sum

If the sum of the values of children equals the value of the parent for extensive quantities. For example, a Categorization containing the Countries in the EU and the EU could set total_sum = True, because the emissions of all parts of the EU must equal the emissions of the EU. On the contrary, a categorization of Industries with categories Power:Fossil Fuels and Power:Gas which are both children of Power must set total_sum = False to avoid double counting of fossil gas.

Type:

bool

canonical_top_level_category

The level of a category is calculated with respect to the canonical top level category. Commonly, this will be the world total or a similar category. If the canonical top level category is not set (i.e. is None), levels are not defined for categories.

Type:

HierarchicalCategory

ancestors(cat: str | HierarchicalCategory) → set[HierarchicalCategory]

All ancestors of the given category, i.e. the direct parents and their parents, etc.

children(cat: str | HierarchicalCategory) → list[set[HierarchicalCategory]]

The list of sets of direct children of the given category.

descendants(cat: str | HierarchicalCategory) → set[HierarchicalCategory]

All descendants of the given category, i.e. the direct children and their children, etc.

property df: DataFrame

All category codes as a pandas dataframe.

extend(*, categories: None | dict[str, dict] = None, alternative_codes: None | dict[str, str] = None, children: None | list[tuple] = None, name: str, title: None | str = None, comment: None | str = None, last_update: None | date = None) → HierarchicalCategorization

Extend the categorization with additional categories and relationships, yielding a new categorization.

Metadata: the name, title, comment, and last_update are updated automatically (see below), the institution and references are deleted and the values for version, hierarchical, total_sum, and canonical_top_level_category are kept. You can set more accurate metadata (for example, your institution) on the returned object if needed.

Parameters:

• categories (dict, optional) – Map of new category codes to their specification. The specification is a dictionary with the keys “title”, optionally “comment”, and optionally “alternative_codes”.

• alternative_codes (dict, optional) – Map of new alternative codes. A dictionary with the new alternative code as key and existing code as value.

• children (list, optional) – List of (parent, (child1, child2, …)) pairs. The given relationships will be inserted in the extended categorization.

• name (str) – The name of your extension. The returned Categorization will have a name of “{old_name}_{name}”, indicating that it is an extension of the underlying Categorization.

• title (str, optional) – A string to add to the original title. If not provided, “ + {name}” will be used.

• comment (str, optional) – A string to add to the original comment. If not provided, “ extended by {name}” will be used.

• last_update (datetime.date, optional) – The date of the last update to this extension. Today will be used if not provided.

Returns:

Extended categorization

Return type:

HierarchicalCategorization

classmethod from_spec(spec: dict[str, Any]) → CategorizationT

Create Categorization from a Dictionary specification.

is_leaf(cat: str | HierarchicalCategory) → bool

Is the category a leaf category, i.e. without children?

items() → ItemsView[str, HierarchicalCategory]

Iterate over (primary code, category) pairs.

leaf_children(cat: str | HierarchicalCategory) → list[set[HierarchicalCategory]]

The sets of subcategories which are descendants of the category and do not have children themselves.

Sets of children are chased separately, so each set of leaf children is self-sufficient to reconstruct this category (if the categorization allows reconstructing categories from their children, i.e. if total_sum is set).

level(cat: str | HierarchicalCategory) → int

The level of the given category.

The canonical top-level category has level 1 and its children have level 2 etc.

To calculate the level, first only the first (“canonical”) set of children is considered. Only if no path from the canonical top-level category to the given category can be found all other sets of children are considered to calculate the level.

parents(cat: str | HierarchicalCategory) → set[HierarchicalCategory]

The direct parents of the given category.

show_as_tree(*, format_func: ~typing.Callable[[~climate_categories._categories.HierarchicalCategory], str] = <class 'str'>, maxdepth: None | int = None, root: None | ~climate_categories._categories.HierarchicalCategory | str = None) → str

Format the hierarchy as a tree.

Starting from the given root, or - if no root is given - the top-level categories (i.e. categories without parents), the tree of categories that are transitive children of the root is show, with children connected to their parents using lines. If a parent category has one set of children, the children are connected to each other and the parent with a simple line. If a parent category has multiple sets of children, the sets are connected to parent with double lines and the children in a set are connected to each other with simple lines.

Parameters:

• format_func (callable, optional) – Function to call to format categories for display. Each category is formatted for display using format_func(category), so format_func should return a string without line breaks, otherwise the tree will look weird. By default, str() is used, so that the first code and the title of the category are used.

• maxdepth (int, optional) – Maximum depth to show in the tree. By default, goes to arbitrary depth.

• root (HierarchicalCategory or str, optional) – HierarchicalCategory object or code to use as the top-most category. If not given, the whole tree is shown, starting from all categories without parents.

Returns:

tree_str – Representation of the hierarchy as formatted string. print() it for optimal viewing.

Return type:

str

to_spec() → dict[str, Any]

Turn this categorization into a specification dictionary ready to be written to a yaml file.

Returns:

spec – Specification dictionary understood by from_spec.

Return type:

dict

values() → ValuesView[HierarchicalCategory]

Iterate over the categories.

class climate_categories.HierarchicalCategory(codes: tuple[str], categorization: HierarchicalCategorization, title: str, comment: None | str = None, info: None | dict = None)

Bases: Category

A single category from a HierarchicalCategorization.

property ancestors: set[HierarchicalCategory]

The super-categories where this category or any of its parents is a member of any set of children, transitively.

Note that all possible ancestors are returned, not only “canonical” ones.

property children: list[set[HierarchicalCategory]]

The sets of subcategories comprising this category.

The first set is canonical, the other sets are alternative. Only the canonical sets are used to calculate the level of a category.

property descendants: set[HierarchicalCategory]

The sets of subcategories comprising this category directly or indirectly.

Note that all possible descendants are returned, not only “canonical” ones.

property is_leaf: bool

Is this category a leaf category, i.e. without children?

property leaf_children: list[set[HierarchicalCategory]]

The sets of subcategories which are descendants of this category and do not have children themselves.

Sets of children are chased separately, so each set of leaf children is self-sufficient to reconstruct this category (if the categorization allows reconstructing categories from their children, i.e. if total_sum is set).

property level: int

The level of the category.

The canonical top-level category has level 1 and its children have level 2 etc.

To calculate the level, only the first (“canonical”) set of children is considered for intermediate categories.

property parents: set[HierarchicalCategory]

The super-categories where this category is a member of any set of children.

Note that all possible parents are returned, not “canonical” parents.

to_spec() → tuple[str, dict[str, str | dict | list]]

Turn this category into a specification ready to be written to a yaml file.

Returns:

(code – Primary code and specification dict

Return type:

str, spec: dict)

climate_categories.find_code(code: str) → set[Category]

Search for the given code in all included categorizations.

climate_categories.from_pickle(filepath: str | Path | IO[bytes]) → Categorization | HierarchicalCategorization

De-serialize Categorization or HierarchicalCategorization from a file written by to_pickle.

Note that this uses the pickle module, which executes arbitrary code in the provided file. Only load from pickle files that you trust.

climate_categories.from_python(filepath: str | Path | IO[bytes]) → CategorizationT

Read Categorization or HierarchicalCategorization from a python cache file.

Note that this executes the python cache file. Only load from python cache files you trust.

climate_categories.from_spec(spec: dict[str, Any]) → CategorizationT

Create Categorization or HierarchicalCategorization from a dict specification.

climate_categories.from_yaml(filepath: str | Path | TextIO) → CategorizationT

Read Categorization or HierarchicalCategorization from a StrictYaml file.

climate_categories.search module

climate_categories.search.search_code(code: str, cats: Iterable[Categorization]) → set[Category]

Search for the given code in the given categorizations.