o DATA STORES (Flat files, RDBMS, Cloud Databases, Big Data / Hadoop):
- DATA CATALOGING OF THE DATA LAKE on local file servers (POSIX file systems), over the cloud (e.g. Amazon S3), on big data clusters (e.g. Hadoop distributions), on messages databases (e.g. Kafka), or on NoSQL databases (e.g. MongoDB):
with high level data lake directory inventory import bridges such as:
· File System (for Linux and Windows) import bridge
· Amazon S3 (Simple Storage Service) import bridge
· Apache Hadoop Distributed File System (HDFS) (for Cloudera, Hortonworks, and MapR) import bridge
· Microsoft Azure Blob Storage import bridge
· Microsoft Azure Data Lake Storage import bridge
· OpenStack Swift (Object Storage) import bridge
· Apache Kafka Cluster import bridge
· MongoDB Database (JSON) import bridge
· Apache CouchDb Database (JSON) import bridge
· MarkLogic NoSQL Database (for JSON only) import bridge
automatically crawling and discovering lower level metadata from flat file, NoSQL, and other file import bridges such as:
· Flat File (CSV) import bridge
· Microsoft Office Excel (XSLX) File import bridge
· JSON (JavaScript Object Notation) File import bridge
· Apache Avro File import bridge
· Apache Parquet File import bridge
· Apache ORC File import bridge
· W3C XML File import bridge
-
DATA SERVICES (REST API, Web Services):
New Bridges:
· Open API Specifications (OAS) import bridge
- OTHER DATA STORES:
New Bridges:
· Apache Hadoop HBase Database (Java API) import bridge
· Apache Hadoop Hive Database (Hive Metastore via JDBC) import bridge
· Microsoft Azure SQL Database import bridge
· Microsoft Azure SQL Data Warehouse import bridge
· SAS Library import bridge
· Snowflake Database (via JDBC) import bridge
o DATA INTEGRATION (DI/ETL/ELT/Scripts):
- New Bridges:
· Apache Sqoop import bridge
· Informatica Developer import new support for DI/ETL bridge
· SAP Data Services (BODI/BODS) import bridge
· SAS base SAS code import bridge
· SAS Data Integration (via SAS Metadata Server) import bridge
· Talend Data Preparation import bridge
- Improved Bridges:
· Microsoft SQL Server Integration Services (SSIS) import support for up to version 14.0 (2017) bridge
o BUSINESS INTELLIGENCE (BI/OLAP):
- New Bridges:
· SAS BI Visual Analytics (via SAS Metadata Server) import bridge
· Microsoft Azure Power BI import bridge
· TIBCO Spotfire Server import bridge
· TIBCO Spotfire Document (File) import bridge
· TIBCO Spotfire Binary Data (SBDF Format) (File) import bridge
· TIBCO Spotfire Text Data (STDF Format) (File) import bridge
· ThoughtSpot import bridge
- Improved Bridges:
· Microsoft SQL Server Analysis Services (SSAS) import support for up to version 14.0 (2017) and specifically its new JSON serialization of the Tabular Model bridge
· Qlik Sense import bridge
o DATA MODELING (DM) & other Modeling Technologies:
- New Bridges:
· W3C Semantic Web Ontology (OWL/RDF) import bridge
o METADATA MANAGEMENT (MM) & DATA GOVERNANCE (DG):
- Improved Bridges:
· Apache Atlas (for HortonWorks) import bridge
· Cloudera Navigator import bridge
o BUSINESS APPLICATIONS:
- Improved Bridges:
· Salesforce (SFDC) import bridge
· SAP Business Suite 4 HANA (S/4HANA), ECC, R/3 ERP import bridge
· SAP Business Warehouse 4 HANA (BW/4HANA) import bridge
o GENERAL:
- All Java code is now compiled with Java 8 without Java 7 compatibility.
- All Third-Party & Open Source Software have been upgraded to their latest versions for better security vulnerability protection.
- Some bridges require access to third-party drivers in their parameters (such as "Google BigQuery Database" or "Microsoft Azure Blob Storage"). In order to facilitate the user experience, some of these bridges are now "Driver Live Download Enabled" where the bridges uses Apache Maven to download the proper driver version and use it automatically. In such case, new messages "Downloading https://repo.maven.apache.org/maven2/" will appear in the bridge log. If the bridge cannot access internet, a message "Error downloading data from URL" is displayed, and the user needs to manually download the driver and provide it as a bridge parameter. The list of bridges that can automatically download drivers from Apache Maven have a download icon on the supported tools page.
o END OF LIFE (EOL) / DEPRECATED BRIDGES:
- Previous FLAT FILE prototype/beta import bridges such as "Flat File (CSV or XLSX) - Beta Bridge" (FlatExcelFile) and "Flat File (CSV) - Beta Bridge" (FlatFile) have been deprecated (EOL) and replaced by the (much more powerful and universal) officially supported import bridge from "File System (CSV, Excel, XML, JSON, Avro, Parquet, ORC)" (FileSystem) bridge
- Previous HADOOP HDFS prototype/beta import bridges such as "Apache Hadoop Distributed File System (WebHDFS REST API) - Beta Bridge" (ApacheHDFSWeb) have been deprecated (EOL) and replaced by the officially supported import bridge from "Apache Hadoop Distributed File System (HDFS Java API)" (ApacheHDFS) bridge