2022.1 (Released on January 26, 2022)

Overview

Indexima’s newest version – v2022.1 – is now available. This new version brings the following main new features:

Easy to use

Table memory footprint: Extension of the SHOW TABLECOUNT function in order to display all the attributes of the tables, as well as more detailed information on the state of the table, the rows awaiting commit, or the state of the tombstone memory of hyperindexes. This allows for a better understanding of the memory footprint and the need to run a COMPACT operation on the table.
Query memory footprint: new query filtering capabilities to quickly identify memory-consuming queries or subqueries.
Simplification of the synchronization process in external table mode: Warning message in case of non-consistency of table structures before starting any synchronization.

Easy to adopt

Denodo is now supported in external tables mode. It enables you to access Denodo as if it was a table in the database.
Data Catalog with Apache Atlas: Improve import process and support deletion in Atlas of non-existing items to simplify integration and bootstrap of catalog.
Extend Microsoft ADLS support: Support for ADLS Gen2 Blob Storage for Indexima warehouse and file load.
Extend CEPH support: Support for CEPH for Indexima warehouse and file load.
Pushdown on AWS Redshift: Support for pushdown functions on AWS Redshift / Postgres allowing the delegation of specific functions to Indexima.

Easy To Implement

Re-initialization process: Accelerate the process of resizing the cluster and reducing the risk of "split-brain" in case of failure detection and cluster reset.
Startup check: Improvement of the startup consistency checks and notifications, to alert your Indexima admin in case of incorrect partitioning parameters.
License is not part of the installation process anymore.

Useful links

Download version 2022.1 (Full version number is 2022.1.1669.x for service pack x)
Migrate from 2021.5 to 2022.1
Procedure for upgrading Indexima Core Engine (Galactica)
Procedure for upgrading Indexima Console (VisualDoop2)

End of life support policy

Please note that releases 1.7.11 and 1.7.12 reached the end of life support as described by our end of life policy.

Detailed Release Note

Reminder: End of Static mode

Since version 2021.5, the static mode (using parameter node in the galactica.conf) is deprecated. Refer to install-indexima-engine for setup details.

Reminder: New user interface

By 2021, we have released a new user interface for the Indexima console. The old User interface is now deprecated.

License

From now on, the only way to update or implement a license is to use the user interface: see here.

Enhancements

INDXMA-6718: Apply a limit to improve JDBC load assistant for Oracle & SQLServer
INDXMA-6920/ INDXMA-6954 : Extend Atlas import function to support deleteNonExisting flag (improve error handling during the import operation) : see Data Catalog with Apache Atlas
INDXMA-6927: Improve query duration measurement (query duration now explicitly includes the initialization time is separated)
INDXMA-6950: Drop support of Indexima license in the conf directory
INDXMA-6953: Optimise diskBackedResult mechanism
INDXMA-6984 (ZenD #1539): Allows to create Alias with the same name as another alias in another schema
INDXMA-7013: Add logs (in galactica.log files) to monitor tables prefetch mechanism
INDXMA-7066: Filter org.apache.thrift.transport.TSaslTransportException expected errors in logs
INDXMA-7020: Extend SHOW TABLECOUNT with new attributes: see SHOW TABLECOUNT
INDXMA-7084: Add lines_after_commit and index_tombstone_percent to SHOW TABLECOUNT
INDXMA-7047 (ZenD #1335): Check Hive table structure before executing a synchronize
INDXMA-7052: Update pre-join Indexes before executing SYNCHRONIZE
INDXMA-7205 (ZenD #1692): Improve error message during a SYNCHRONIZE operation
INDXMA-7071: Improve syntax control of Synchronize with partitions
INDXMA-7125: Fix cases of missing backquotes for schema beginning with a figure
INDXMA-7162: Better handle BIT type used to represent boolean in PostgreSQL
INDXMA-7169: Improve node synchronization for pending queries
INDXMA-7171: Update supported_dbs.json.template with new sources
INDXMA-7181: Speed-up hive initialization for Hadoop3 installation
INDXMA-7201: Add new configuration files (atlas-application.properties & core-site.xml) to the diagnostic Zip file
INDXMA-7207: Refresh the tablecount of a table after a COMPACT operation
INDXMA-7211: Prevent creating two equivalent hyperindexes
INDXMA-7275: Allows to load from a local filesystem on windows nodes
INDXMA-7156: Improve support for loading files from ADLS gen2 blob storage

Improving queries executions

INDXMA-7158: Optimisation to evaluate only once some constant expressions
INDXMA-7135: Optimise index selection for subqueries
INDXMA-6963: Optimise DROP SCHEMA CASCADE with a parallelization deletion process
INDXMA-7265: Catch exception upon query completion to prevent never-ending queries

Improving Cluster Stability, cluster recovery & Node Status Management

INDXMA-7019: Migrate re-init operation on the new network stack
INDXMA-7029: Prevent unrequired node failure event upon cluster re-init
INDXMA-7005: Rework channel management for ping service
INDXMA-7131: Optimise warehouse reload upon re-init
INDXMA-7114: Simplify ping monitoring
INDXMA-7182: Migrate STATUS operation to the new network stack
INDXMA-7212: Improve error message when a query is canceled due to cluster in error
INDXMA-7224: Migrate UPDATE LICENSE operation on the new network stack
INDXMA-7237: Improve status reporting upon node lost
INDXMA-7127: Enhance cluster status management and re-init process
INDXMA-7263: Review granularity of new Node Status in logs

Configuration

INDXMA-6908: Remove parameter tableau_cast_timestamp
INDXMA-6956: Remove parameter warehouse.protocol
INDXMA-7035: Simplify Indexima console connexion files
INDXMA-7048: Improve startup control of partition parameter misaligned with existing warehouse
INDXMA-7259: Improve notification upon the misaligned tables with partition parameter
INDXMA-7198: Clean versions in DeprecatedProperties

External Table

New supported connections

INDXMA-6930: Support of external table on sap Hana
INDXMA-7087: Support external tables on Denodo

New Supported elements

~~INDXMA-6928: Fix the JBDC URL connexion string when using the indexima assistant to create a table from sap Hana (~~
INDXMA-6967: Allows to delegate NaN as DOUBLE
INDXMA-6968: Allows to delegate NULL as a timestamp
INDXMA-6990: Support dense_rank as a pushdown function on external tables
INDXMA-7007: Fix delegation of "when is not null" expression
INDXMA-7163: Support Unhex for external tables on PostgreSQL
INDXMA-6960: Support Postgres/Redshift pushdown functions
INDXMA-6961: Support Oracle pushdown functions

Improving Queries executions

INDXMA-6944: Better handle session pool when delegating to external tables
INDXMA-6965: Fix error when creating an external table on Indexima if schema specified in URL
INDXMA-7039: Killed delegated queries are not closed on Impala source

Improving Synchronization process

INDXMA-7228: Add option for non-incremental Snowflake synchro
INDXMA-7050: Prevent triggering twice a SYNCHRONIZE on the same table
INDXMA-7051: Flush snowflake stream upon SYNCHRONIZE ALL
INDXMA-7061: Raise an error when adding a pre-join index on an external snowflake table if there is no corresponding index on joined tables

Misc

INDXMA-7102: Handle column names with spaces when delegating to SQL server or sap Hana
~~INDXMA-7103: Fix delegation errors when spaces in column names in the source~~
~~INDXMA-7106: Fix load assistant incorrect parsing for external tables on SAP Hana~~
INDXMA-7107: Improve escaping table names when delegating to SAP Hana
INDXMA-7138: Fix cases of incorrect external table selection with dimension tables
INDXMA-7176: Improve escaping alias names when delegating to SAP Hana
INDXMA-7217: Fix an incorrect quoting when creating an external table through the "table creation" wizard
INDXMA-7291: Fix error when creating a table from PostgreSQL using the "table creation" wizard

Smart Table, Analyzer, and Index generation

INDXMA-6969: Beware of NULL constants when generating string representing indexes
INDXMA-6993: Add the default optimizer mode after the first run of the analyzer
INDXMA-7060: Automatically generate an index on joined tables when adding a pre-join index on an external table
INDXMA-7074: Extend analyzer API to specify if analyzer triggered an index on another dimension table
INDXMA-7110: Log current user when using the Analyzer shortcut buttons
INDXMA-7129: Fix exception when running a Quick Analysis on schema beginning with a figure
INDXMA-7155: Fix exception when parsing query history
INDXMA-7188: Fix timeout for advanced analyzer
INDXMA-7192: Fix incorrect alias renaming when generating index
INDXMA-7204: Fix analyzer suggestion incompatible with SAP Hana dialect
INDXMA-7216: Improve cardinality evaluation logic for small tables
INDXMA-7297: Extend timeout when triggering analyzer through dev console

Security

INDXMA-7101: Activate API secret key check even if Hive authentication deactivated
INDXMA-7148: Vulnerability CVE-2021-44228 - Update log4j
INDXMA-7185: Vulnerability CVE-2021-45105 & CVE-2021-45046 - Update to log4j 2.17.0
INDXMA-7227 (ZenD #1669): Vulnerability CVE-2021-44832 - Update to log4j 2.17.1

Indexima Console

Major improvements

INDXMA-6270: Migrate application to Vue CLI https://cli.vuejs.org/
INDXMA-6796: Decommissioning old dev console
INDXMA-6987: Fix missing compression on some endpoints

Cluster connexions Hub

INDXMA-7031: Fix usage of MAIN_CLUSTER_IP and MAIN_CONNECTION_URL
INDXMA-6929: Persist visualdoop token to keep user sessions upon service restart
INDXMA-7036: Rework timeout of API request when updating a cluster or connexion
INDXMA-7059: Fix wrong connection label
INDXMA-7082: Replace deprecated node-saas
INDXMA-7100: Catch console error upon opening an invalid cluster
INDXMA-7122: Catch incorrect connection errors on visualdoop startup
INDXMA-7133: Save button spinning indefinitely when testing incorrect cluster host
INDXMA-7274: Remove deprecated High Availability from cluster homepage

Improving Queries View

INDXMA-7093: Add load index flag in query listing
INDXMA-6849: Add filter on used memory in query listing
INDXMA-7080: Display query initialization_duration in the console
INDXMA-7143: Fix incorrect information in queries details upon several quick restarts
INDXMA-7145: Fix query listing expand/collapse mechanism
INDXMA-7298: Add index used filter on queries view
INDXMA-7094: Improve calculation of MAX_WORKER_MEMORY for subqueries

Misc

INDXMA-6837: Improve Notifications service
INDXMA-6936 (ZenD #1540): Fix upgrade license through the WebUI may not work
INDXMA-6949: Solve Javascript error if license expired
INDXMA-6974: Fix catalog error when trying to open a deleted table
INDXMA-6999: Display column names even if zero results
INDXMA-7004: Improve timezone support with new endpoints for queries & events
INDXMA-7027: Redirect from load assistant goes to the cluster view
INDXMA-7034: Fix routing issue when displaying nodes logs
INDXMA-7049: Improve query profile fields description
INDXMA-7097: Fix missing workbooks in the console after migration
INDXMA-7109 (ZenD #1644): Rework Biggest Indexes tooltips
INDXMA-7123: Fix console incorrect accent display
INDXMA-7126: Display Data change the position of action buttons
INDXMA-7128: Fix cases of missing backquotes when using the 'Copy in Editor' button
INDXMA-7140: Extend timeout for the diagnosis download
INDXMA-7166: Fix WebSocket connection failed in console
INDXMA-7179: Fix Azure synapse and MS SQL logo collision
INDXMA-7180: Improve load assistant for Netezza and Denodo
INDXMA-7187: Upgrade node-status page according to new re-init stack
INDXMA-7220: Fix creating dimension table from JDBC load assistant
INDXMA-7230: Improve display of table names in the catalog
INDXMA-7232: Fix Boolean support in the console
INDXMA-7258: Sort by node-id on the node status page
INDXMA-7262: Fix potential server error on cluster status endpoint
INDXMA-7264: Improve close button for tabs on console view
INDXMA-7294: Speed up the display of the analyzer notifications
INDXMA-7295: Fix analyzer display during analyze / loading

Bugs

INDXMA-6977: Fix conversion of count distinct dictionary
INDXMA-6986 (ZenD #1532): Fix error when the subquery is multinode whereas parent query is a one-node query
INDXMA-6989: Fix exception when delegating a UNION
INDXMA-7003: Fix exception when creating a table from AS SELECT
INDXMA-7032 (ZenD #1609): Fix missing version in dev console for windows deployment
INDXMA-7038: Fix hadoop3 control triggering database does not exist when loading from self
INDXMA-7056: Fix concurrent access error while adding an index
INDXMA-7085: Inconsistent behavior of union when a subquery is executed on a secondary master
INDXMA-7096: Improve timestamp cast when loading index
INDXMA-7108 (ZenD #1643): Add quotes to better handle scheme name starting with a figure
INDXMA-7111 (ZenD #1555): Fix edge case of Having function not applied
INDXMA-7117 (ZenD #1617): Fix cases of incorrect rotation of queriescsv logs
INDXMA-7120: Fix a missing compression when generating the diagnostic file
INDXMA-7124 (ZenD #1577): Fix evaluation of NULL timestamps
INDXMA-7146 (ZenD #1647): Fix null pointer error when merging an index with a count distinct
INDXMA-7149: Fix load error on very large lines
INDXMA-7151 (ZenD #1647): Fix potential index corruption when compacting an hyperindex containing a count distinct aggregation
INDXMA-7189: Fix computation of PreviousWeekNum
INDXMA-7210: Fix case of null pointer exception on the analyzer incremental store
INDXMA-7225 (ZenD #1693 #1694): Fix Timestamp conversion error after migrating to 2021.5
INDXMA-7153: Fix left outer join without ON clause