End-​to-end automation of the corporate credit process

Not all digital is the same

Depending on which input channel is selected, different process steps follow (printout, physical storage, storage of attachments, storage of uploads). The problem with the classic, non-automated process, however, is the manual, error-prone capture, the possibly local storage of personal data, the lack of a digital workflow following capture, and the fact that data is not available for further analysis across all borrowers or reporting.

At the end of automated processing, therefore, the goal is always to convert  the content of a document into a machine-readable and interpretable form, ideally in the form of a database entry. This is where the potential of process automation really comes into its own.

At the same time, not all digital is the same, which brings us to the question of the degree of digitalization and the types of documents.

Degree of digitalization and types of documents

Documents in paper form, i.e., analog, are naturally the furthest away from the desired degree of digitalization. The first step here is the time-consuming and costly scanning, which is why it is in the interest of the credit institution to have this processing step performed by the submitter, where possible. However, scanners create raster graphics, i.e., images divided into pixels with different color and brightness values per pixel (measured in HSL (hue, lightness and saturation, for example). The same applies to images submitted in JPG format, for example.

The error-free conversion of the pixels into digital characters, predominantly in UTF-8 Unicode format, is therefore a particular challenge. This is especially true when handwriting recognition or the elimination of smudges and text overlays are also involved.

The msg group offers a workable solution for this with its Smart Input Management msg.SIMA, which can also be used as software as a service (SaaS) without cost-intensive on-premise installation. The data is made available in XML format for further processing.

However, PDF documents are often created directly on the computer from a file. In this case, the information contained in the file can be read directly. It is even easier with Word and Excel and CSV formats, which have already been digitized and are comparatively simple to process further.

In addition to the degree of digitalization, however, documents are also structured differently, which influences the complexity of the reading and interpretation process.

From structured, semi-structured and unstructured data

The simplest case is completely structure documents; when the credit institution provides a form whose fields can already be defined in a document capture solution. This way, borrowers could be requested to provide certain basic information in a form. Other documents, such as an extract from the commercial register, are also predominantly structured in the same way and are therefore relatively easy to read.

Semi-​structured documents contain essentially the same information, but differ significantly in terms of content and structure. For example, business reports can use different charts of accounts, contain company-specific accounts and sub-accounts and include attachments. The challenge is then to anticipate these different cases as far as possible in order to be able to transfer the financial data obtained into an automated balance sheet and P&L analysis.

The class of largely or completely unstructured documents could already include annual financial statements. Although they still follow a certain form with their components balance sheet, income statement, notes to the financial statements and management report, they already differ significantly depending on the size and purpose of the company. Special information required for automated balance sheet and P&L analysis can also be located in different places in the annual financial statements.

An example of this is current liabilities with a remaining term of less than one year. On the one hand, they can appear directly in the balance sheet, and on the other hand, they can also appear in further detail in a statement of liabilities. The challenge then is to identify the relevant places in the overall document and to read them in without errors.

msg GillardonBSM AG has already developed solutions for reading in structured and semi-structured documents based on Python scripts that communicate with the process engine. Libraries such as Numpy (math and vector library), Pandas (database and table library) and PDFPlumber (library for easy reading and processing of PDF files) are used.

In the simplest case, a table can be read out completely using predefined functions. Only the formatting of the text and individual cells is necessary. This method is used for reading the extracts from the trade register. If tables do not have conventional frame lines, a table must be formed manually. Here, algorithms are used to calculate and estimate cell boundaries that build tables from information about word positions and display width/height.

Process-related processing

Process-related processing can be divided into three main steps:

Read-in process:

• Conversion of different formats, structures and content into a machine-readable form (database table) using a wide range of character sets.
• PDF files are read in fully automatically; other file formats of input documents are converted into a PDF format and read in (*.txt, *.jpg, *.tiff, Excel).
• Optical Character Recognition (OCR) can be used to scan multi-page documents, including handwritten ones, and identify pages that belong together.

Quality assurance:

• With appropriate parameterization, usually 80 – 85 percent of the data contained can be read in error-free and reliably. Plausibility checks, lists of accepted values and data training help to reduce the susceptibility to errors.
• Promising 100% automation, however, would not be a serious approach, as there may also be dubious cases that require intervention at the human-machine interface, such as documents that are dirty, bent, or overlaid with handwritten remarks. It is also possible that the documents are the wrong ones.
• For quality assurance purposes, it is therefore necessary to display of doubtful or rejected documents for manual assessment. Here, a traffic light system can visualize a check routine that classifies doubtful or rejected documents.

Interpretation:

With the conversion into a machine-readable and thus analyzable form, the essential prerequisites for leveraging the potential in loan process automation have been met.

However, with the help of artificial intelligence (AI), further benefits can be achieved in the next step. AI is always used when it is a matter of generating new knowledge from existing data, provided that we  want to go beyond purely deterministic approaches. In the context of input documents, text mining is particularly worth mentioning here. Under the keywords Natural Language Processing and Natural Language Understanding, this involves the cognitive capture and semantic analysis of content using machine learning methods.

Incoming documents are therefore not simply read, but also interpreted. This can be useful when analyzing documents from specific industry information services, for sentiment analysis, or generally for evaluating unstructured documents, such as those stored in NoSQL databases like MongoDB or CouchDB. Wordclouds and histograms are suitable for visualization.

In addition to the widely used libraries and frameworks, such as Tensorflow, PyTorch or scikit-learn, there are also special libraries such as Gensim, NLTK or SpaCy for the analysis of semantic structures and keywords.

Business Case

The metric for calculating the benefits of automated document capture differs from bank to bank depending on the parameters used, such as quantity structure, employee capacity used, or personnel costs. In principle, however, a number of key points can be defined.

On the expense side are:

• Implementation costs (installation, customizing and training), which are, however, reduced or made more variable by a SaaS solution
• Usage and license fees, which are usually based on the number of documents read per year in the form of package prices

This is offset on the income side by:

• Reduction of employee capacity required for manual data capture
• Simplification, acceleration and stabilization of the entire process

In the form of a dynamic payback calculation, taking into account the time value, the initial capital investment, the annual return on capital and a discount factor, the benefits can be presented very well in measurable and verifiable quantities.

Conclusion

Finally, let's summarize the functionalities and benefits again: 

Functionalities

• Minimal manual intervention required for document reading
• Self-learning system – results improve on the fly
• Flexible input options and a wide range of supported document formats
• Simple and efficient system maintenance
• Target output fully configurable – preparation for further processing possible
• On-premise operation or in the cloud
• Integration into existing business processes or stand-alone system

Benefits

• Cost savings
• Free up valuable time for more value-adding activities
• Quality-assured input for further processing
• Optimization of lead times in the loan process
• Integration with upstream and downstream systems
• Connection of workflows
• Basis for consistent reporting
• Traceability for auditing purposes