TMW-API Preprocessors

API-Version: 1.0.2

Fileversion: 2020-11-24

For simple help see here.
For detailed documentation see here.

Using the Processors

Automatic call

All preprocessors can be called automatically when retrieving data from a certain domain. In this case, the call is set in the config.ini. For each domain a list of preprocessors can be defined, which are executed in the order of the list in the config file.

	; config.ini entry in the processor section
	[processor]
		domain1[] = {processor_name1}[.{parameter1}]
		domain1[] = {processor_name2}[.{parameter2}]
		...
		domain2[] = {processor_name}[.{parameter}]
		...
	

HTTP call

The second way to call a preprocessor is adding the processor name (with an optional parameter) after the domain name in the http request:

It is possible to call a list of preprocessors by separating the name-parameter with a pipe charachter:

In any case the processor list from the http request is worked off after the predefined list in the config.ini file.

Multiple parameters

Not yet implemented

If a processor supports multiple parameters to be given, this is done by separating the values with the pipe character |.

Processors

Link checker

The linkcheck processor looks for broken internal links to SolR datasets. If the linked dataset does not exist, the entry in the controlled dataset is removed. The linkcheck needs a parameter to define the domain to which the links will be checked:

	; definition in the config.ini
	[processor]
	{domain1} = link.{domain2}

	; call from the http request
	http://root/domain1/link.{domain2}/query...

	

For each domain a {domain}.ini file can be created, containing the information about the links to be checked.

	; link check settings in the {domain}.ini file of the link check postprocessor
	[{field_to_check}]
		domain = "{domain_name}" ; domain of the link target
		field = "{field_name}" ; field name of the referenced field
		reffield = "{ref_field_name}" ; field name of the reference number in the checked record
		lref = "{ref_field_name_lref}" ; lref of the reffield

	[...]
		...

	

Link resolver

The link resolver is used to get detailed information from internal linked records. Resolving data returns field information, which is inserted as a linkedData group in the corresponding entry. If the inserted block contains extern open data link, these are resolved by the open data resolver.

	<broader_term>
		{term}

	; added data from hierarchy processor
		<linkedData>
			<pageName>Technisches Museum Wien</pageName>
			<id>{broader_term_id}</id>
			<term>{broader_term}</term>
			<scopeNote>{broader_term_scopenote}</scopeNote>

			; optional external open data links
			[<link>{open_data_links}</link>]
		</linkedData>
	</broader_term>
	...
	

Open data resolver

The resolver is used to read certain data from a open data link directly to the output from the api. The resolver has a plugin interface to handle different api concepts. The block with the resolved data always has the same format, not depending on the source. <linkedData> blocks can be nested, if a extern link is included in an intern resolved link. In this case, the link resolver has to be called before the open data resolver. There are the following plugins available:

• DNB - Gemeinsame Normdatei der Deutschen Nationalbibliothek
• Geonames - geonames.org
• Mediawiki - all wiki platforms using the mediawiki framework (wikipedia)

The resolve preprocessor is simply called by adding resolve as the preprocessor. In this case a <linkedData> tag is added to the <link> tag of the SolR data :

	<link>
		{link_name}

	; added data from resolve processor
		<linkedData>
			<pageName>{name_of_the_open_data_platform}</pageName>
			<id>{page_ID}</id>
			<term>{page_title}</term>
			<scopenote>{content_summary}</scopenote>
		</linkedData>
	</link>
	

With the additional parameter the internal links inside the open data resource, as well as the external links to other resources can be added too.

• ilinks - adding internal links
• elinks - adding external links
• links - adding all links

	http://root/domain/resolve/query... ; resolve data
	http://root/domain/resolve.links/query... ; resolve data with all links
	

	<link>
		<text> ; link with optional additional parameters

			<uri>{link_name}</uri>

	; added data from resolve processor
			<linkedData>
				<pageName>{name_of_the_open_data_platform}</pageName>
				<id>{page_ID}</id>
				<term>{page_title}</term>
				<scopenote>{content_summary}</scopenote>

				<ilinks type="list">
					<link>
						<url>{page_url}<url>
						<title>{page_title}<title>
					<link>
					...
				</ilinks>

				<elinks type="list">
					<link>
						<url>{page_url}<url>
						<title>{page_title}<title>
					<link>
					...
				</elinks>

			</linkedData>
		</text>

		<text> ; link with optional additional parameters
			...
		</text>

	</link>
	

Topterm

Still in developement

The topterm preprocessor can follow a hierarchy and add the records linked to the starting data. The content of a field can be used to define the topterm to stop the walk process. If no end condition is defined, the walk process continues to the record without a further link.

When the processor is called, additional records are added to the starting record in the order of the hierarchical linking. The parameters for the walk process are layed down in a .ini file, named by the domain: {domain}.ini.

The ini file contains three parameters:

	linkfield = "{record_field_to_be_followed}"
	endfield = "{record_field_for_end_condition}"
	endvalue = "{end_field_value_for_walk_end}"