Release Notes

Release Notes #

Information about release notes of INFINI Gateway is provided here.

1.6.0 #

Breaking changes #

  • Update disk_queue folder structure, use UUID as folder name instead of the queue name
  • Parameter mode was removed from bulk_reshuffle filter, only async was supported
  • Rename filter bulk_response_validate to bulk_response_process

Features #

  • Add metadata to queue
  • Support subscribe queue by specify labels
  • Support concurrent worker control for bulk_indexing processor
  • Auto detect new queues for bulk_indexing processor
  • Allow to consume queue messages over disk queue
  • Auto sync disk_queue files to remote s3 in background
  • Add api to operate gateway entry
  • Support plugin auto discovery
  • Add API to operate gateway entities
  • Filter bulk_request_mutate support remove _type in bulk requests for es v8.0+
  • Add elasticsearch adapter for version 8.0+
  • Add http filter for general reverse proxy usage, like proxy Kibana
  • Add consumer_has_lag condition to check queue status
  • Add record filter to play requests easier
  • Add zstd compress to disk_queue, disabled by default

Bug fix #

  • Fix date_range_precision_tuning filter for complex range query
  • Fix node availability initially check
  • Fix basic_auth filter not asking user to input auth info in browser

Improvements #

  • Handle http public address, remove prefix if that exists
  • Refactor bulk_reshuffle filter and bulk_indexing processor
  • Should not fetch nodes info when elasticsearch discovery disabled
  • Seamless consume queue message across files
  • Persist consumer offset to local store
  • Add API to reset consumer offset
  • Refactoring ORM framework
  • Expose error of mapping put
  • Refactoring pipeline framework
  • Improve multi-instance check, multi-instance disabled by default
  • Add CPU and memory metrics to stats api
  • Seamless fetch queue files from s3 server
  • Proper handle 409 version conflicts in bulk requests

1.5.0 #

Breaking changes #

Features #

  • Add API to scroll messages from disk queue
  • Prevent out of space, disk usage reserved for disk_queue
  • Add context_filter and context_limiter for general purpose
  • Add bulk_request_mutate filter
  • Add basic_auth filter
  • Add set_context filter
  • Add context_regex_replace filter
  • Add to_string property to request and response context

Bug fix #

  • Fix bulk response validate incorrectly caused by jsonParser
  • Fix nil exception in request_path_limiter caused by refactoring
  • Fix big size document out of order caused by bulk buffer

Improvements #

  • Fix TCP not keepalived in some case
  • Add closing progress bar to pipeline module
  • Add retry_delay_in_ms config to pipeline module
  • Handle partial failure in bulk requests
  • Optimize scroll performance of dump_hash processor
  • Improve API directory

1.4.0 #

Breaking changes #

  • Rename flow config filter_v2 to filter, only support new syntax
  • Rename pipeline config pipelines_v2 to pipeline, processors to processor, only support new syntax
  • Rename filter request_logging to logging
  • Merge dump filters to dump filter
  • Response headers renamed, dashboard may broken
  • Remove filter request_body_truncate and response_body_truncate

Features #

  • Add option to disable file logging output
  • Add option compress to queue_consumer processor

Bug fix #

  • Fix invalid host header setting in elasticsearch reverse proxy
  • Fix cluster available health check
  • Fix gzip encoding issue for requests forwarding

Improvements #

  • Support string type in in condition

1.3.0 #

Breaking changes #

  • Switch to use pipelines_v2 syntax only
  • Rename filter disk_enqueue to queue
  • Rename processor disk_queue_consumer to queue_consumer
  • Rename filter redis to redis_pubsub

Features #

  • Refactoring pipeline framework, support DAG based task schedule
  • Add dump_hash and index_diffs processor
  • Add redis output and redis queue adapter
  • Add set_request_query_args filter
  • Add ldap_auth filter
  • Add retry_limiter filter
  • Add request_body_json_set and request_body_json_del filter
  • Add stats filter
  • Add health_check config to elastic module
  • Add API to pipeline framework, support _start and _stop pipelines

Bug fix #

  • Fix data race issue in bulk_reshuffle
  • Fix fix_null_id always executed in bulk_reshuffle
  • Auto handle big sized documents in bulk requests

Improvements #

  • Refactoring flow runner to service pipeline
  • Optimize CPU and Memory usage
  • Optimize index diff service, speedup and cross version compatibility
  • Set the default max file size of queue files to 1 GB
  • Proper handle elasticsearch failure during startup
  • Support custom depth check to queue_has_lag condition
  • Support multi hosts for elasticsearch configuration
  • Add parameter auto_start to prevent pipeline running on start
  • Add keep_running parameter to pipeline config
  • Safety shutdown pipeline and entry service
  • Support more complex routing pattern rules

1.2.0 #

Features #

  • Support alias in bulk_reshuffle filter.
  • Support truncate in request_logging filter.
  • Handle 429 retry in json_indexing service.
  • Add forcemerge service.
  • Add response_body_regex_replace filter.
  • Add request_body_regex_replace filter.
  • Add sleep filter.
  • Add option to log slow requests only.
  • Add cluster and bulk status to request logging.
  • Add filter_v2 and support _ctx to access request context.
  • Add dump_context filter.
  • Add translog filter, support rotation and compression.
  • Add set_response filter.
  • Add set_request_header filter.
  • Add set_hostname filter.
  • Add set_basic_auth filter.
  • Add set_response_header filter.
  • Add elasticsearch_health_check filter.
  • Add drop filter.

Bug fix #

  • Fix truncate body filter, correctly resize the body bytes.
  • Fix cache filter.
  • Fix floating_ip module.
  • Fix dirty write in diskqueue.
  • Fix compression enabled requests.
  • Fix date_range_precision_tuning filter.
  • Fix invalid indices status on closed indices #23.
  • Fix document hash for elasticsearch 6.x.
  • Fix floating_ip feature run with daemon mode.
  • Fix async bulk to work with beats.

Improvements #

  • Optimize memory usage, fix memory leak.

Acknowledgement #

Thanks to the following enterprises and teams #

  • China Everbright Bank, China Citic Bank, BSG, Yogoo

Thanks to the following individual contributors #

  • MaQianghua, YangFan, Tanzi, FangLi

1.1.0 #

  • Request Logging and Dashboard.
  • Support ARM Platform [armv5\v6\v7\v8(arm64)].
  • Fix Elasticsearch Nodes Auto Discovery.
  • Add Request Header Filter.
  • Add Request Method Filter.
  • Add Sample Filter.
  • Request Logging Performance Optimized (100x speedup).
  • Add Request Path Filter.
  • Add Debug Filter.
  • Add User Info to Logging Message.
  • Support Routing Partial Traffic to Specify Processing Flow (by Ratio).
  • Support Traffic Clone, Support Dual-Write or 1:N Write.
  • Elasticsearch topology auto discovery, support filter by nodes,tags,roles.
  • Backend failure auto detection, auto retry and select another available endpoint.
  • Floating IP feature ready to use.
  • Add bulk_reshuffle filter.

1.0.0 #

  • Rewritten for performance
  • Index level request throttle
  • Request caching
  • Kibana MAGIC speedup
  • Upstream auto discovery
  • Weighted upstream selections
  • Max connection limit per upstream
  • Languages English
©INFINI.LTD, All Rights Reserved.