ES Release Notes 5.0.0 ~ 5.6.0

memo-release-notes-5.0.0-alpha1-2x.md

5.0.0-alpha1 Release Notes (Changes previously released in 2.x)

The changes listed below were first released in the 2.x series. Changes released for the first time in Elasticsearch 5.0.0-alpha1 are listed in 5.0.0-alpha1 Release Notes.

Breaking changes

Index APIs

• Change Field stats API response to include both number and string based min and max values #14674 (issue: #14404)
• Add Force Merge API, deprecate Optimize API #13778

Internal

• Forbid changing thread pool types #14367 (issues: #14294, #2509, #2858, #5152)

Logging

• Log cluster health status changes #14557 (issue: #11657)

Mapping

• Add per-index setting to limit number of nested fields #15989 (issue: #14983)

Nested Docs

• If sorting by nested field then the nested_path should always be specified #13429 (issue: #13420)

Scripting

• Filter classes loaded by scripts #15262
• Lock down javascript and python script engines better #13924

Search

• Limit the size of the result window to a dynamic property #13188 (issue: #9311)

Breaking Java changes

Allocation

• Speed up shard balancer by reusing shard model while moving shards that can no longer be allocated to a node #16926

New features

Aggregations

• Adds geo_centroid metric aggregator #13846 (issue: #13621)
• Add percentiles_bucket pipeline aggregation #13186
• Add stats_bucket / extended_stats_bucket pipeline aggs #13128

Geo

• Add CONTAINS relation to geo_shape query #14810 (issue: #14713)
• Add support for Lucene 5.4 GeoPoint queries #14537
• Add GeoPointV2 Field Mapping #14536

Network

• Allow binding to multiple addresses. #13954 (issue: #13592)

Plugin Analysis Phonetic

• Add support for daitch_mokotoff #14834

Plugin Cloud AWS

• Add support for S3 storage class #13656 (issue: #13655)

Plugins

• Decentralize plugin security #14108

Search

• Add query profiler #14889 (issues: #12974, #6699)

Enhancements

Aliases

• Add support to _aliases endpoint to specify multiple indices and aliases in one action #15305 (issue: #15186)

Allocation

• Skip capturing least/most FS info for an FS with no total #16001 (issue: #15919)
• Speed improvements for BalancedShardsAllocator #15678 (issue: #6372)
• Simplify delayed shard allocation #14808
• Add cluster-wide setting for total shard limit #14563 (issue: #14456)
• Early terminate high disk watermark checks on single data node cluster #13882 (issue: #9391)
• Also use PriorityComparator in shard balancer #13256 (issue: #13249)
• Add support for filtering by publish IP address #8801

Analysis

• Add detail response support for _analyze API #11660 (issue: #11076)

CAT API

• Add sync_id to cat shards API #14712 (issue: #14705)
• Add duration field to /_cat/snapshots #14385
• Add cat API for repositories and snapshots #14247 (issue: #13919)
• Adds disk used by indices to _cat/allocation #13783 (issue: #13529)

Cluster

• Shard state action request logging #16396
• Safe cluster state task notifications #15777
• Reroute once per batch of shard failures #15510
• Add callback for publication of new cluster state #15494 (issue: #15482)
• Use general cluster state batching mechanism for shard started #15023 (issues: #14725, #14899)
• Use general cluster state batching mechanism for shard failures #15016 (issues: #14725, #14899)
• Set an newly created IndexShard’s ShardRouting before exposing it to operations #14918 (issue: #10708)
• Uniform exceptions for TransportMasterNodeAction #14737

Core

• Remove log4j exception hiding #16834
• Avoid cloning MessageDigest instances #16479
• Add a hard check to ensure we are running with the expected lucene version #16305 (issue: #16301)
• If we can’t get a MAC address for the node, use a dummy one #15266 (issue: #10099)
• Simplify shard inactive logging #15259 (issue: #15252)
• Simplify IndexingMemoryController#checkIdle #15252 (issue: #15251)
• IndexingMemoryController should not track shard index states #15251 (issues: #13918, #15225)
• Verify Checksum once it has been fully written to fail as soon as possible #13896

Discovery

• Don’t allow nodes with missing custom meta data to join cluster #15401 (issue: #13445)

Exceptions

• Added file name to exceptions when failing to read index state #16850 (issue: #16713)
• Add Exception class name to message in NotSerializableExceptionWrapper #16325
• Deduplicate cause if already contained in shard failures #14432
• Give a better exception when running from freebsd jail without enforce_statfs=1 #14135 (issue: #12018)
• Make root_cause of field conflicts more obvious #13976 (issue: #12839)
• Use a dedicated id to serialize EsExceptions instead of it’s class name. #13629

Fielddata

• Update GeoPoint FieldData for GeoPointV2 #14345

Geo

• Upgrade GeoPointField to use Lucene 5.5 PrefixEncoding #16482
• Geo: Fix toString() in GeoDistanceRangeQuery and GeoPolygonQuery #15026
• Enable GeoPointV2 with backward compatibility testing #14667 (issues: #10761, #11159, #9859)
• Refactor Geo utilities to Lucene 5.4 #14339

Index APIs

• Add option to disable closing indices #14169 (issue: #14168)

Index Templates

• Disallow index template pattern to be the same as an alias name #15184 (issue: #14842)

Internal

• Implement available for all StreamInput classes #17218
• Cleanup search sub transport actions and collapse o.e.action.search.type package into o.e.action.search #16758 (issue: #11710)
• Simplify the Text API. #15511
• Simpler using compressed oops flag representation #15509 (issue: #15489)
• Info on compressed ordinary object pointers #15489 (issues: #13187, #455)
• Explicitly log cluster state update failures #15428 (issues: #14899, #15016, #15023)
• Use transport service to handle RetryOnReplicaException to execute replica action on the current node #15363
• Make IndexShard operation be more explicit about whether they are expected to run on a primary or replica #15282
• Avoid trace logging allocations in TransportBroadcastByNodeAction #15221
• Only trace log shard not available exceptions #14950 (issue: #14927)
• Transport options should be immutable #14760
• Fix dangling comma in ClusterBlock#toString #14483
• Improve some logging around master election and cluster state #14481
• Add System#exit(), Runtime#exit() and Runtime#halt() to forbidden APIs #14473 (issue: #12596)
• Simplify XContent detection. #14472
• Add threadgroup isolation. #14353
• Cleanup plugin security #14311
• Add workaround for JDK-8014008 #14274
• Refactor retry logic for TransportMasterNodeAction #14222
• Remove MetaDataSerivce and it’s semaphores #14159 (issue: #1296)
• Cleanup IndexMetaData #14119
• Add SpecialPermission to guard exceptions to security policy. #13854
• Clean up scripting permissions. #13844
• Factor groovy out of core into lang-groovy #13834 (issue: #13725)
• More helpful error message on parameter order #13737
• Factor expressions scripts out to lang-expression plugin #13726 (issue: #13725)
• Cleanup InternalClusterInfoService #13543
• Remove and forbid use of com.google.common.base.Throwables #13409 (issue: #13224)
• Remove cyclic dependencies between IndexService and FieldData / BitSet caches #13381
• Remove and forbid use of com.google.common.base.Objects #13355 (issue: #13224)
• Enable indy (invokedynamic) compile flag for Groovy scripts by default #8201 (issue: #8184)

Java API

• Add created flag to IndexingOperationListener#postIndex #17340 (issue: #17333)
• Align handling of interrupts in BulkProcessor #15527 (issue: #14833)
• BulkProcessor backs off exponentially by default #15513 (issue: #14829)
• Reject refresh usage in bulk items when using and fix NPE when no source #15082 (issue: #7361)
• BulkProcessor retries after request handling has been rejected due to a full thread pool #14829 (issue: #14620)

Logging

• Log suppressed stack traces under DEBUG #16627 (issues: #12991, #15329, #16622)
• Add circuit breaker name to logging package #14661
• Move logging for the amount of free disk to TRACE #14403 (issue: #12843)
• Map log-level trace to JDK-Level FINEST #14234

Mapping

• Expose the reason why a mapping merge is issued. #16059 (issue: #15989)
• Add sub-fields support to bool fields. #15636 (issue: #6587)
• Improve cross-type dynamic mapping updates. #15633 (issue: #15568)
• Make mapping updates more robust. #15539
• Make mapping serialization more robust. #15480
• Make mappings immutable. #15313 (issue: #9365)
• Make MappedFieldType.checkTypeName part of MappedFieldType.checkCompatibility. #15245
• Register field mappers at the node level. #14896 (issue: #14828)

Network

• Provide better error message when an incompatible node connects to a node #17182 (issue: #17090)
• Add additional fallback to http.publish_port and restrict fallback to transport.publish_port #16626 (issue: #14535)
• only allow code to bind to the user’s configured port numbers/ranges #14549
• Port of publishAddress should match port of corresponding boundAddress #14535 (issues: #14503, #14513, #14514)

Packaging

• Windows service: Use JAVA_HOME environment variable in registry #16552 (issue: #13521)
• Default standard output to the journal in systemd #16159 (issues: #15315, #16134)
• Use egrep instead of grep -E for Solaris #15755 (issue: #15628)
• punch thru symlinks when loading plugins/modules #15311
• set ActiveProcessLimit=1 on windows #15055
• set RLIMIT_NPROC = 0 on bsd/os X systems. #15039
• Drop ability to execute on Solaris #14200
• Nuke ES_CLASSPATH appending, JarHell fail on empty classpath elements #13880 (issues: #13812, #13864)
• improve seccomp syscall filtering #13829
• Block process execution with seccomp on linux/amd64 #13753
• Get lang-javascript, lang-python, securemock ready for script refactoring #13695
• Remove some bogus permissions only needed for tests. #13620
• Remove java.lang.reflect.ReflectPermission "suppressAccessChecks" #13603
• Remove JAVA_HOME detection from the debian init script #13514 (issues: #13403, #9774)

Plugin Cloud AWS

• Add ap-northeast-2 (seoul) endpoints for EC2 discovery and S3 snapshots #16167 (issue: #16166)
• Add aws canned acl #14297 (issue: #14103)
• Improved building of disco nodes #14155
• Enable S3SignerType #13360 (issue: #13332)

Plugin Cloud Azure

• Add support for secondary azure storage account #13779 (issue: #13228)

Plugin Cloud GCE

• cloud-gce plugin should check discovery.type #13809 (issue: #13614)
• Adding backoff from retries on GCE errors #13671 (issue: #13460)

Plugin Discovery EC2

• Adding US-Gov-West #14358

Plugin Ingest Attachment

• Fix attachments plugins with docx #17059 (issue: #16864)

Plugins

• Expose http.type setting, and collapse al(most all) modules relating to transport/http #15434 (issue: #14148)
• Ban RuntimePermission("getClassLoader") #15253
• Add nicer error message when a plugin descriptor is missing #15200 (issue: #15197)
• Don’t be lenient in PluginService#processModule(Module) #14306
• Adds a validation for plugins script to check if java is set #13633 (issue: #13613)
• Output plugin info only in verbose mode #12908 (issue: #12907)

Query DSL

• Allow CIDR notation in query string query #14773 (issue: #7464)
• Internal: simplify filtered query conversion to lucene query #13312 (issue: #13272)

REST

• Make XContentGenerator.writeRaw* safer. #15358
• Filter path refactoring #14390 (issues: #10980, #11560, #13344)

Recovery

• Handle cancel exceptions on recovery target if the cancel comes from the source #15309
• Decouple routing and primary operation logic in TransportReplicationAction #14852

Reindex API

• Implement helpful interfaces in reindex requests #17032
• Reindex should timeout if sub-requests timeout #16962
• Teach reindex to retry on rejection #16556 (issue: #16093)

Scripting

• Remove suppressAccessChecks permission for Groovy script plugin #16839 (issue: #16527)
• Class permission for Groovy references #16660 (issue: #16657)
• Scripting: Allow to get size of array in mustache #16193
• Enhancements to the mustache script engine #15661
• Add property permissions so groovy scripts can serialize json #14500 (issue: #14488)
• Remove ScriptEngineService.unwrap. #13958
• Remove ScriptEngineService.execute. #13956

Search

• Caching Weight wrappers should propagate the BulkScorer. #14317
• fix numerical issue in function score query #14085
• Optimize scrolls for constant-score queries. #13311

Settings

• Log warning if max file descriptors too low #16506

Snapshot/Restore

• Support wildcards for getting repositories and snapshots #15151 (issue: #4758)
• Add ignore_unavailable parameter to skip unavailable snapshot #14471 (issue: #13887)
• Simplify the BlobContainer blob writing interface #13434

Stats

• Pull Fields instance once from LeafReader in completion stats #15090 (issue: #6593)
• Add os.allocated_processors stats #14409 (issue: #13917)
• Adds stats counter for failed indexing requests #13130 (issue: #8938)

Top Hits

• Put method addField on TopHitsBuilder #14597 (issue: #12962)

Translog

• Check for tragic event on all kinds of exceptions not only ACE and IOException #15535

Tribe Node

• Tribe nodes should apply cluster state updates in batches #14993 (issues: #14725, #14899)

Bug fixes

Aggregations

• Prevents exception being raised when ordering by an aggregation which wasn’t collected #17379 (issue: #17225)
• Setting other bucket on empty aggregation #17264 (issue: #16546)
• Build empty extended stats aggregation if no docs collected for bucket #16972 (issues: #16812, #9544)
• Set meta data for pipeline aggregations #16516 (issue: #16484)
• Filter(s) aggregation should create weights only once. #15998
• Make missing on terms aggs work with all execution modes. #15746 (issue: #14882)
• Run pipeline aggregations for empty buckets added in the Range Aggregation #15519 (issue: #15471)
• [Children agg] fix bug that prevented all child docs from being evaluated #15457
• Correct typo in class name of StatsAggregator #15321 (issue: #14730)
• Fix significant terms reduce for long terms #14948 (issue: #13522)
• Fix NPE in Derivative Pipeline when current bucket value is null #14745
• Pass extended bounds into HistogramAggregator when creating an unmapped aggregator #14742 (issue: #14735)
• Added correct generic type parameter on ScriptedMetricBuilder #14018 (issue: #13986)
• Pipeline Aggregations at the root of the agg tree are now validated #13475 (issue: #13179)

Aliases

• Fix _aliases filter and null parameters #16553 (issues: #16547, #16549)

Allocation

• IndicesStore checks for allocated elsewhere for every shard not alocated on the local node #17106
• Prevent peer recovery from node with older version #15775
• Fix calculation of next delay for delayed shard allocation #14765
• Take ignored unallocated shards into account when making allocation decision #14678 (issue: #14670)
• Only allow rebalance operations to run if all shard store data is available #14591 (issue: #14387)
• Delayed allocation can miss a reroute #14494 (issues: #14010, #14011, #14445)
• Check rebalancing constraints when shards are moved from a node they can no longer remain on #14259 (issue: #14057)

Analysis

• Analysis : Allow string explain param in JSON #16977 (issue: #16925)
• Analysis : Fix no response from Analyze API without specified index #15447 (issue: #15148)

Bulk

• Bulk api: fail deletes when routing is required but not specified #16675 (issues: #10136, #16645)
• Do not release unacquired semaphore #14909 (issue: #14908)

CAT API

• Fix column aliases in _cat/indices, _cat/nodes and _cat/shards APIs #17145 (issue: #17101)
• Properly set indices and indicesOptions on subrequest made by /_cat/indices #14360

CRUD

• Throw exception if content type could not be determined in Update API #15904 (issue: #15822)
• Index name expressions should not be broken up #13691 (issue: #13665)

Cache

• Handle closed readers in ShardCoreKeyMap #16027

Cluster

• Index deletes not applied when cluster UUID has changed #16825 (issue: #11665)
• Only fail the relocation target when a replication request on it fails #15791 (issue: #15790)
• Handle shards assigned to nodes that are not in the cluster state #14586 (issue: #14584)
• Bulk cluster state updates on index deletion #11258 (issue: #7295)

Core

• BitSetFilterCache duplicates its content. #15836 (issue: #15820)
• Limit the max size of bulk and index thread pools to bounded number of processors #15585 (issue: #15582)
• AllTermQuery’s scorer should skip segments that never saw the requested term #15506
• Include root-cause exception when we fail to change shard’s index buffer #14867
• Restore thread interrupt flag after an InterruptedException #14799 (issue: #14798)
• Record all bytes of the checksum in VerifyingIndexOutput #13923 (issues: #13848, #13896)
• When shard becomes active again, immediately increase its indexing buffer #13918 (issue: #13802)
• Close TokenStream in finally clause #13870 (issue: #11947)
• LoggingRunnable.run should catch and log all errors, not just Exception? #13718 (issue: #13487)

Exceptions

• Fix ensureNodesAreAvailable’s error message #14007 (issue: #13957)

Expressions

• Check that _value is used in aggregations script before setting value to specialValue #17091 (issue: #14262)

Fielddata

• Don’t cache top level field data for fields that don’t exist #14693

Geo

• Remove .geohash suffix from GeoDistanceQuery and GeoDistanceRangeQuery #15871 (issue: #15179)
• Geo: Allow numeric parameters enclosed in quotes for geohash_grid aggregation #14440 (issue: #13132)
• Resync Geopoint hashCode/equals method #14124 (issue: #14083)
• Fix GeoPointFieldMapper to index geohash at correct precision. #13649 (issue: #12467)

Highlighting

• Don’t override originalQuery with request filters #15793 (issue: #15689)
• Fix spans extraction to not also include individual terms. #15516 (issues: #13239, #15291)

Index APIs

• Field stats: Index constraints should remove indices in the response if the field to evaluate is empty #14868
• Field stats: Fix NPE for index constraint on empty index #14841
• Field stats: Added format option for index constraints #14823 (issue: #14804)
• Forbid index name . and .. #13862 (issue: #13858)

Inner Hits

• Query and top level inner hit definitions shouldn’t overwrite each other #16222 (issue: #16218)

Internal

• Log uncaught exceptions from scheduled once tasks #15824 (issue: #15814)
• FunctionScoreQuery should implement two-phase iteration. #15602
• Make sure the remaining delay of unassigned shard is updated with every reroute #14890 (issue: #14808)
• Throw a meaningful error when loading metadata and an alias and index have the same name #14842 (issue: #14706)
• fixup issues with 32-bit jvm #14609
• Failure to update the cluster state with the recovered state should make sure it will be recovered later #14485
• Gateway: a race condition can prevent the initial cluster state from being recovered #13997
• Verify actually written checksum in VerifyingIndexOutput #13848
• An inactive shard is activated by triggered synced flush #13802
• Remove all setAccessible in tests and forbid #13539
• Remove easy uses of setAccessible in tests. #13537
• Ban setAccessible from core code, restore monitoring stats under java 9 #13531 (issue: #13527)

Logging

• Add missing index name to indexing slow log #17026 (issue: #17025)
• ParseFieldMatcher should log when using deprecated settings. #16988
• Don’t log multi-megabyte guice exceptions. #13782
• Moving system property setting to before it can be used #13660 (issue: #13658)

Mapping

• Put mapping operations must update metadata of all types. #16264 (issue: #16239)
• Fix serialization of search_analyzer. #16255
• Reuse metadata mappers for dynamic updates. #16023 (issue: #15997)
• Fix MapperService#searchFilter(…) #15923 (issue: #15757)
• Fix initial sizing of BytesStreamOutput. #15864 (issue: #15789)
• MetaDataMappingService should call MapperService.merge with the original mapping update. #15508
• MapperService: check index.mapper.dynamic during index creation #15424 (issue: #15381)
• Only text fields should accept analyzer and term vector settings. #15308
• Mapper parsers should not check for a tokenized property. #15289
• Validate that fields are defined only once. #15243 (issue: #15057)
• Check mapping compatibility up-front. #15175 (issue: #15049)
• Don’t treat default as a regular type. #15156 (issue: #15049)
• Don’t ignore mapping merge failures. #15144 (issue: #15049)
• Treat mappings at an index-level feature. #15142
• Make _type use doc values #14783 (issue: #14781)

Network

• Only accept transport requests after node is fully initialized #16746 (issue: #16723)

Packaging

• Fix waiting for pidfile #16718 (issue: #16717)
• Fix Windows service installation failure #15549 (issue: #15349)
• Enable es_include at init #15173
• Handle system policy correctly #14704 (issue: #14690)
• Startup script exit status should catch daemonized startup failures #14170 (issue: #14163)
• Don’t let ubuntu try to install its crazy jayatana agent. #13813 (issue: #13785)

Parent/Child

• Check that parent_type in Has Parent Query has child types #16923 (issue: #16692)
• Has child query forces default similarity #16611 (issues: #16550, #4977)

Percolator

• Don’t replace found fields if map unmapped fields as string is enabled #16043 (issue: #10500)
• mpercolate api should serialise start time #15938 (issue: #15908)

Plugin Delete By Query

• Fix Delete-by-Query with Shield #14658 (issue: #14527)

Plugin Discovery GCE

• Add setFactory permission to GceDiscoveryPlugin #16860 (issue: #16485)

Plugin Mapper Attachment

• Fix toXContent() for mapper attachments field #15110

Plugin Repository S3

• Hack around aws security hole of accessing sun.security.ssl, s3 repository works on java 9 again #13538 (issue: #432)

Plugins

• Fix plugin list command error message #14288 (issue: #14287)
• Fix HTML response during redirection #11374 (issue: #11370)

Query DSL

• Fix FunctionScore equals/hashCode to include minScore and friends #15676
• Min should match greater than the number of optional clauses should return no result #15571 (issue: #15521)
• Return a better exception message when regexp query is used on a numeric field #14910 (issue: #14782)

REST

• Remove detect_noop from REST spec #16386
• Make text parsing less lenient. #15679
• Throw exception when trying to write map with null keys #15479 (issue: #14346)
• Fix OOM in AbstractXContentParser #15350 (issue: #15338)
• XContentFactory.xContentType: allow for possible UTF-8 BOM for JSON XContentType #14611 (issue: #14442)
• RestUtils.decodeQueryString ignores the URI fragment when parsing a query string #13365 (issue: #13320)

Recovery

• Try to renew sync ID if flush=true on forceMerge #17108 (issue: #17019)
• CancellableThreads should also treat ThreadInterruptedException as InterruptedException #15318

Reindex API

• Properly register reindex status #17125
• Make search failure cause rest failure #16889 (issue: #16037)

Scripting

• Add permission to access sun.reflect.MethodAccessorImpl from Groovy scripts #16540 (issue: #16536)
• Security permissions for Groovy closures #16196 (issues: #16194, #248)

Search

• Do not apply minimum_should_match on auto generated boolean query if the coordination factor is disabled. #16155
• Do not apply minimum-should-match on a boolean query if the coords are disabled #16078 (issue: #15858)
• Fix blended terms take 2 #15894 (issue: #15860)
• Fix NPE when a segment with an empty cache gets closed. #15202 (issue: #15043)
• Fix the quotes in the explain message for a script score function without parameters #11398

Settings

• TransportClient should use updated setting for initialization of modules and service #16095
• ByteSizeValue.equals should normalize units #13784

Snapshot/Restore

• Prevent closing index during snapshot restore #16933 (issue: #16321)
• Add node version check to shard allocation during restore #16520 (issue: #16519)
• Snapshot restore and index creates should keep index settings and cluster blocks in sync #13931 (issue: #13213)
• Fix blob size in writeBlob() method #13574 (issue: #13434)

Stats

• Fix recovery translog stats totals when recovering from store #16493 (issue: #15974)
• Fix calculation of age of pending tasks #15995 (issue: #15988)
• Add extra validation into cluster/stats #14699 (issue: #7390)
• Omit current* stats for OldShardStats #13801 (issue: #13386)

Task Manager

• Fix TaskId#isSet to return true when id is set and not other way around #17307
• Don’t wait for completion of list tasks tasks when wait_for_completion flag is set #17231

Translog

• Never delete translog-N.tlog file when creation fails #15788
• Close recovered translog readers if createWriter fails #15762 (issue: #15754)
• Fail and close translog hard if writing to disk fails #15420 (issue: #15333)
• Prevent writing to closed channel if translog is already closed #15012 (issue: #14866)
• Don’t delete temp recovered checkpoint file if it was renamed #14872 (issue: #14695)
• Translog recovery can repeatedly fail if we run out of disk #14695
• Pending operations in the translog prevent shard from being marked as inactive #13759 (issue: #13707)

Tribe Node

• Passthrough environment and network settings to tribe client nodes #16893
• Tribe node: pass path.conf to inner tribe clients #16258 (issue: #16253)
• Fix tribe node to load config file for internal client nodes #15300 (issues: #13383, #14573)

Regressions

Analysis

• Add PathHierarchy type back to path_hierarchy tokenizer for backward compatibility with 1.x #15785 (issue: #15756)

Internal

• Deduplicate concrete indices after indices resolution #14316 (issues: #11258, #12058)

Plugin Cloud Azure

• Filter cloud azure credentials #14863 (issues: #13779, #14843)

REST

• Don’t return all indices immediately if count of expressions >1 and first expression is * #17033 (issue: #17027)

Upgrades

Core

• Upgrade to Lucene 5.5.0 official release #16742
• Upgrade to lucene 5.5.0-snapshot-850c6c2 #16615
• Upgrade to lucene 5.5.0-snapshot-4de5f1d #16400 (issues: #16373, #16399)
• Update lucene to r1725675 #16114
• Upgrade to lucene-5.5.0-snapshot-1721183. #15575
• Upgrade Lucene to 5.4.0-snapshot-1715952 #14951
• Upgrade Lucene to 5.4.0-snapshot-1714615 #14784
• Upgrade to lucene-5.4.0-snapshot-1712973. #14619
• update to lucene-5.4.x-snapshot-1711508 #14398
• Upgrade to lucene-5.4-snapshot-1710880. #14320
• Upgrade to lucene-5.4-snapshot-1708254. #14074
• upgrade lucene to r1702265 #13439
• Upgrade master to lucene 5.4-snapshot r1701068 #13324

Geo

• Update to spatial4j 0.5 for correct Multi-Geometry #14269 (issue: #9904)

Internal

• Update to Jackson 2.6.2 #13344 (issues: #10980, #207, #213)

Plugin Cloud AWS

• Update AWS SDK version to 1.10.19 #13655 (issue: #13656)

Plugin Cloud Azure

• Upgrade Azure Storage client to 4.0.0 #16084 (issues: #12567, #15080, #15976)
• Update Azure Service Management API to 0.9.0 #15232 (issue: #15209)

Plugin Discovery Azure

• Upgrade azure SDK to 0.9.3 #17102 (issues: #17042, #557)

Plugin Discovery EC2

• Upgrade to aws 1.10.33 #14672

Plugin Lang JS

• upgrade rhino for plugins/lang-javascript #14466

Scripting

• Upgrade groovy dependency in lang-groovy module to version 2.4.6 #16830 (issue: #16527)

memo-release-notes-5.0.0-alpha1.md

5.0.0-alpha1 Release Notes

The changes listed below have been released for the first time in Elasticsearch 5.0.0-alpha1. Changes in this release which were first released in the 2.x series are listed in 5.0.0-alpha1 Release Notes (Changes previously released in 2.x).

Also see Breaking changes in 5.0.

This is an alpha release and is intended for testing purposes only. Indices created in this version will not be compatible with Elasticsearch 5.0.0 GA. Upgrading 5.0.0-alpha1 to any other version is not supported.

Breaking changes

Aliases

• make get alias expand to open and closed indices by default #15954 (issue: #14982)
• Remove deprecated indices.get_aliases #13906

Allocation

• Remove DisableAllocationDecider #13313

CAT API

• Add raw recovery progress to cat recovery API #17064 (issue: #17022)
• Remove host from cat nodes API #16656 (issues: #12959, #16575)
• Using the accept header in the request instead of content-type in _cat API. #14421 (issue: #14195)

CRUD

• Remove object notation for core types. #15684 (issue: #15388)

Cache

• Remove deprecated query cache settings #15592

Cluster

• Remove memory section #17278 (issues: #12049, #16756)

Core

• Bootstrap does not set system properties #17088 (issues: #16579, #16791)
• Add max number of processes check #16919
• Add mlockall bootstrap check #16909
• One log #16703 (issue: #16585)

Engine

• Remove index.compound_on_flush setting and default to true #15594 (issue: #10778)

Fielddata

• Remove "uninverted" and "binary" fielddata support for numeric and boolean fields. #14082

Index APIs

• Remove GET option for /_forcemerge #15223 (issue: #15165)
• Remove /_optimize REST API endpoint #14226 (issue: #13778)

Internal

• Remove support for pre 2.0 indices #13799

Mapping

• Change the field mapping index time boost into a query time boost. #16900
• Deprecate string in favor of text/keyword. #16877
• Term vector APIs should no longer update mappings #16285
• Remove the format option of the _source field. #15398
• Remove transform #13657 (issue: #12674)

Parent/Child

• Removed total score mode in favour for sum score mode. #17174 (issues: #13470, #17083)
• Removed pre 2.x parent child implementation #13376

Percolator

• Change the percolate api to not dynamically add fields to mapping #16077 (issue: #15751)

Plugins

• Rename bin/plugin in bin/elasticsearch-plugin #16454
• Remove multicast plugin #16326 (issue: #16310)
• Plugins: Remove site plugins #16038

Query DSL

• Function score query: remove deprecated support for boost_factor #13510
• Remove support for deprecated queries. #13418 (issue: #13326)

REST

• Limit the accepted length of the _id #16036 (issue: #16034)

Search

• Remove deprecated reverse option from sorting #17282 (issue: #17047)
• Remove some deprecations #14331
• Remove search exists api #13911 (issues: #13682, #13910)
• Remove the scan and count search types. #13310

Search Refactoring

• Remove "query" query and fix related parsing bugs #14304 (issue: #13326)

Settings

• Remove ability to specify arbitrary node attributes with node. prefix #17402 (issue: #17280)
• Enforce discovery.zen.minimum_master_nodes is set when bound to a public ip #17288
• Prevent index level setting from being configured on a node level #17144 (issue: #16799)
• Remove support for node.client setting #16963 (issue: #16565)
• Remove es.max-open-files flag #16757 (issues: #16506, #483)
• Enforce node level limits if node is started in production env #16733 (issue: #16727)
• Make settings validation strict #16365
• Remove the ability to fsync on every operation and only schedule fsync task if really needed #16257 (issue: #16152)
• Script settings #16197
• Remove index.flush_on_close entirely #15977
• Restore chunksize of 512kb on recovery and remove configurability #15235 (issue: #15161)
• Remove ancient deprecated and alternative recovery settings #15234
• Fix ping timeout settings inconsistencies #13701 (issue: #6579)

Similarities

• Renames default similarity into classic #15446 (issue: #15102)

Snapshot/Restore

• Fail closing or deleting indices during a full snapshot #17021 (issue: #16321)

Stats

• Modify load average format #15932 (issue: #15907)
• Reintroduce five-minute and fifteen-minute load averages on Linux #15907 (issues: #12049, #14741)
• Add system CPU percent to OS stats #14741

Term Vectors

• Remove DFS support from TermVector API #16452

Translog

• Drop support for simple translog and hard-wire buffer to 8kb #15574
• Simplify translog-based flush settings #15573

Warmers

• Remove query warmers and the warmer API. #15614 (issue: #15607)

Breaking Java changes

Aggregations

• getKeyAsString and key_as_string should be the same for terms aggregation on boolean field #15393

Allocation

• Simplify shard balancer interface #17028 (issue: #8954)

Analysis

• Simplify Analysis registration and configuration #14355

Cache

• Refactor IndicesRequestCache to make it testable. #16610
• Fold IndexCacheModule into IndexModule #14293

Core

• Remove es.useLinkedTransferQueue #16786

Discovery

• Include pings from client nodes in master election #17329 (issue: #17325)

Internal

• Remove duplicate getters from DiscoveryNode and DiscoveryNodes #17410 (issue: #16963)
• Cli: Switch to jopt-simple #17024 (issue: #11564)
• Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation #15776
• Remove NodeBuilder #15354
• Fix IndexSearcherWrapper interface to not depend on the EngineConfig #14654
• Cleanup query parsing and remove IndexQueryParserService #14452
• Remove circular dependency between IndicesService and IndicesStore #14285
• Remove guice injection from IndexStore and friends #14279
• Replace IndicesLifecycle with a per-index IndexEventListener #14217 (issue: #13259)
• Simplify similarity module and friends #13942
• Refactor SearchRequest to be parsed on the coordinating node #13859

Java API

• Remove the count api #14166 (issue: #13928)
• IdsQueryBuilder to accept only non null ids and types #13937

Mapping

• [Mapping] Several MappingService cleanups #16133 (issue: #15924)

Network

• Remove ability to disable Netty gathering writes #16774 (issue: #7811)

Parent/Child

• Cleanup ParentFieldMapper #16045
• Several other parent/child cleanups #13470

Plugins

• Enforce isolated mode for all plugins #17276
• Change the inner structure of the plugins zip #16453
• Don’t use guice for QueryParsers #15761
• Remove guice from the index level #14518
• Remove shard-level injector #13881

Query DSL

• Remove the MissingQueryBuilder which was deprecated in 2.2.0. #15364 (issue: #14112)
• Remove NotQueryBuilder #14204 (issue: #13761)

Search

• Query refactoring: split parse phase into fromXContent and toQuery for all queries #13788 (issue: #10217)

Search Refactoring

• Refactored inner hits parsing and intoduced InnerHitBuilder #17291
• Remove deprecated parameter from field sort builder. #16573 (issue: #16127)
• Remove support for query_binary and filter_binary #14433 (issue: #14308)
• Validate query api: move query parsing to the coordinating node #14384

Settings

• Move remaining settings in NettyHttpServerTransport to the new infra #16531
• Replace IndexSettings annotation with a full-fledged class #14251

Store

• Standardize state format type for global and index level metadata #17123

Suggesters

• Remove suggest threadpool #17304 (issue: #17198)
• Remove suggest transport action #17198 (issue: #10217)

Deprecations

Plugin Mapper Attachment

• Deprecate mapper-attachments plugin #16948 (issue: #16910)

Search

• Deprecate fuzzy query #16211 (issues: #15760, #16121)

New features

Allocation

• Add API to explain why a shard is or isn’t assigned #17305 (issue: #14593)

Discovery

• Add two phased commit to Cluster State publishing #13062

Ingest

• Merge feature/ingest branch into master branch #16049 (issue: #14049)

Mapping

• Add a text field. #16637
• Add a new keyword field. #16589

Percolator

• index the query terms from the percolator query #13646 (issue: #12664)

Plugin Ingest Attachment

• Ingest: Add attachment processor #16490 (issue: #16303)

Plugin Mapper Attachment

• Migrate mapper attachments plugin to main repository #14605

Plugin Repository HDFS

• HDFS Snapshot/Restore plugin #15192 (issue: #15191)

Query DSL

• Adds a rewrite phase to queries on the shard level #16870 (issue: #9526)

Reindex API

• Merge reindex to master #16861

Scripting

• Exceptions and Infinite Loop Checking #15936
• Added a new scripting language (PlanA) #15136 (issue: #13084)

Search

• Add search_after parameter in the SearchAPI #16125 (issue: #8192)

Settings

• Add infrastructure to transactionally apply and reset dynamic settings #15278

Stats

• API for listing index file sizes #16661 (issue: #16131)

Suggesters

• Add document-oriented completion suggester #14410 (issue: #10746)

Task Manager

• Add task cancellation mechanism #16320
• Make the Task object available to the action caller #16033
• Task Management: Add framework for registering and communicating with tasks #15347 (issue: #15117)

Enhancements

Aggregations

• Add tests and documentation for using time_zone in date range aggregation #16955 (issue: #10130)
• Fixes serialisation of Ranges #16674

Allocation

• Write shard state metadata as soon as shard is created / initializing #16625 (issue: #14739)
• Reuse existing allocation id for primary shard allocation #16530 (issue: #14739)
• Remove version in ShardRouting (now obsolete) #16243 (issue: #14739)
• Prefer nodes that previously held primary shard for primary shard allocation #16096 (issue: #14739)
• Extend reroute with an option to force assign stale primary shard copies #15708 (issue: #14739)
• Allocate primary shards based on allocation IDs #15281 (issue: #14739)
• Persist currently started allocation IDs to index metadata #14964 (issue: #14739)
• Use ObjectParser to parse AllocationID #14962 (issue: #14831)
• Persist allocation ID with shard state metadata on nodes #14831 (issue: #14739)

Analysis

• Improve error message if resource files have illegal encoding #17237 (issue: #17212)

CAT API

• Expose http address in cat/nodes #16770
• [cat/recovery] Make recovery time a TimeValue() #16743 (issue: #9209)
• :CAT API: remove space at the end of a line #15250 (issue: #9464)

CRUD

• CRUD: Allow to get and set ttl as a time value/string #15047

Cache

• Enable the indices request cache by default #17162 (issues: #16870, #17134)

Cluster

• Cluster Health should run on applied states, even if waitFor=0 #17440
• Resolve index names to Index instances early #17048
• Remove DiscoveryNode#shouldConnectTo method #16898 (issue: #16815)
• Fail demoted primary shards and retry request #16415 (issue: #14252)
• Illegal shard failure requests #16275
• Shard failure requests for non-existent shards #16089 (issue: #14252)
• Add handling of channel failures when starting a shard #16041 (issue: #15895)
• Wait for new master when failing shard #15748 (issue: #14252)
• Master should wait on cluster state publication when failing a shard #15468 (issue: #14252)
• Split cluster state update tasks into roles #14899 (issue: #13627)
• Add timeout mechanism for sending shard failures #14707 (issue: #14252)
• Add listener mechanism for failures to send shard failed #14295 (issue: #14252)

Core

• Remove PROTOTYPE from BulkItemResponse.Failure #17433 (issue: #17086)
• Throw an exception if Writeable.Reader reads null #17332
• Remove PROTOTYPE from RescorerBuilders #17330
• Port Primary Terms to master #17044 (issues: #14062, #14651, #17038)
• Use index UUID to lookup indices on IndicesService #17001
• Add -XX+AlwaysPreTouch JVM flag #16937
• Add max size virtual memory check #16935
• Use and test relative time in TransportBulkAction #16916
• Bump Elasticsearch version to 5.0.0-SNAPSHOT #16862
• Assert that we can write in all data-path on startup #16745
• Add G1GC check on startup #16737 (issue: #10740)
• Shards with heavy indexing should get more of the indexing buffer #14121
• Remove and ban ImmutableMap #13939 (issue: #13224)
• Finish banning ImmutableSet #13820 (issue: #13224)
• Removes and bans ImmutableSet #13754 (issue: #13224)
• Remove and ban ImmutableMap#entrySet #13724
• Forbid ForwardingSet #13720 (issue: #13224)

Discovery

• Add a dedicate queue for incoming ClusterStates #13303 (issue: #13062)

Engine

• Remove writeLockTimeout from InternalEngine #16930
• Don’t guard IndexShard#refresh calls by a check to isRefreshNeeded #16118
• Never call a listerner under lock in InternalEngine #15786
• Use System.nanoTime() to initialize Engine.lastWriteNanos #14321
• Flush big merges automatically if shard is inactive #14275
• Remove Engine.Create #13955
• Remove the disabled autogenerated id optimization from InternalEngine #13857

Exceptions

• Fix typos in exception/assert/log messages in core module. #16649
• Add field names to several mapping errors #16508 (issue: #16378)
• Add serialization support for more important IOExceptions #15766
• Adds exception objects to log messages. #14827 (issue: #10021)
• Add stack traces to logged exceptions where missing #13825 (issue: #10021)
• Remove reflection hacks from ElasticsearchException #13796
• Rename QueryParsingException to a more generic ParsingException #13631
• Add *Exception(Throwable cause) constructors/ call where appropriate #13544 (issue: #10021)

Geo

• Fix a potential parsing problem in GeoDistanceSortParser #17111
• Geo: Add validation of shapes to ShapeBuilders #15551 (issue: #14416)
• Make remaining ShapeBuilders implement Writeable #15010 (issue: #14416)
• Geo: Remove internal translated flag from LineStringBuilder #14969
• Make PointBuilder, CircleBuilder & EnvelopeBuilder implement Writable #14933 (issue: #14416)
• Merging BaseLineString and BasePolygonBuilder with subclass #14887 (issue: #14482)
• Moving static factory methods to ShapeBuilders #14529
• Remove InternalLineStringBuilder and InternalPolygonBuilder #14482 (issue: #14416)

Highlighting

• Switch Highlighting to ObjectParser #17363
• Use HighlightBuilder in SearchSourceBuilder #15376 (issue: #15044)
• Joint parsing of common global Hightlighter and subfield parameters #15368 (issue: #15285)
• Enable HighlightBuilder to create SearchContextHighlight #15324
• Add fromXContent method to HighlightBuilder #15157

Ingest

• add automatic type conversion support to ConvertProcessor #17263 (issue: #17139)
• Give the foreach processor access to the rest of the document #17172 (issue: #17147)
• Added ingest statistics to node stats API #16915
• Add ingest_took to bulk response #16876
• Add ingest info to node info API, which contains a list of available processors #16865
• Use diffs for ingest metadata in cluster state #16847
• hide null-valued metadata fields from WriteableIngestDocument#toXContent #16557
• Ingest: use bulk thread pool for bulk request processing (was index before) #16539 (issue: #16503)
• Add foreach processor #16432
• revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation #16355
• Add processor tags to on_failure metadata in ingest pipeline #16324 (issue: #16202)
• catch processor/pipeline factory exceptions and return structured error responses #16276 (issue: #16010)
• Ingest: move get/put/delete pipeline methods to ClusterAdminClient #16242
• Geoip processor: remove redundant latitude and longitude fields and make location an object with lat and lon subfields #16173

Internal

• Remove PROTOTYPE from MLT.Item #17481 (issue: #17085)
• Remove PROTOTYPE from VersionType #17480 (issue: #17085)
• Remove PROTOTYPEs from highlighting #17466 (issue: #17085)
• Remove PROTOTYPEs from ingest #17434 (issue: #17085)
• Start to rework query registration #17424
• Factor out slow logs into Search and IndexingOperationListeners #17398
• Remove PROTOTYPE from Suggesters #17370
• Remove PROTOTYPE from SortBuilders #17337 (issue: #17085)
• Remove PROTOTYPE from ShapeBuilders #17336 (issue: #17085)
• Replace FieldStatsProvider with a method on MappedFieldType. #17334
• Stop using PROTOTYPE in NamedWriteableRegistry #17284 (issue: #17085)
• Support scheduled commands in current context #17077
• Thread limits #17003
• Remove leniency from segments info integrity checks #16985 (issue: #16973)
• Rename SearchServiceTransportAction to SearchTransportService #16880
• Decouple the TransportService and ClusterService #16872 (issue: #16788)
• Refactor bootstrap checks #16844 (issues: #16733, #16835)
• Add LifecycleRunnable #16752
• Hot inlined methods in your area #16725
• Move IndicesQueryCache and IndicesRequestCache into IndicesService #16603
• Forbid use of java.security.MessageDigest#clone() #16543 (issue: #16479)
• Make IndicesWarmer a private class of IndexService #16470
• Simplify IndicesFieldDataCache and detach from guice #16469
• Uppercase ells (L) in long literals #16329 (issue: #16279)
• ShardId equality and hash code inconsistency #16319 (issue: #16217)
• Ensure all resources are closed on Node#close() #16316 (issue: #13685)
• Make index uuid available in Index, ShardRouting & ShardId #16217
• Move RefreshTask into IndexService and use since task per index #15933
• Make IndexingMemoryController private to IndicesService #15877
• Cleanup IndexingOperationListeners infrastructure #15875
• Remove and forbid use of j.u.c.ThreadLocalRandom #15862 (issue: #15294)
• Fix IntelliJ query builder type inference issues #15429
• Remove and forbid use of Collections#shuffle(List) and Random#<init>() #15299 (issue: #15287)
• Remove and forbid use of the type-unsafe empty Collections fields #15187
• Move IndicesService.canDeleteShardContent to use IndexSettings #15150 (issue: #15059)
• Simplify MonitorService construction and detach from guice #15035
• Use Supplier for StreamInput#readOptionalStreamable #14806
• Add variable-length long encoding #14780
• Extend usage of IndexSetting class #14731 (issue: #14251)
• Fold SimilarityModule into IndexModule #14284
• Move to lucene BoostQuery #14264
• Use built-in method for computing hash code of longs #14213
• Refactor ShardFailure listener infrastructure #14206
• Add methods for variable-length encoding integral arrays #14087
• Fold IndexAliasesService into IndexService #14044
• Remove unneeded Module abstractions #13944
• Query refactoring: simplify IndexQueryParserService parse methods #13938 (issue: #13859)
• Remove and forbid use of com.google.common.collect.Iterators #13916 (issue: #13224)
• Remove and forbid use of com.google.common.collect.ImmutableCollection #13909 (issue: #13224)
• Remove and forbid use of com.google.common.io.Resources #13908 (issue: #13224)
• Remove and forbid use of com.google.common.hash.* #13907 (issue: #13224)
• Remove and forbid use of com.google.common.net.InetAddresses #13905 (issue: #13224)
• Remove and forbid use of com.google.common.collect.EvictingQueue #13903 (issue: #13224)
• Replace Guava cache with simple concurrent LRU cache #13879
• Remove ClusterSerivce and IndexSettingsService dependency from IndexShard #13853
• Start making RecoverySourceHandler unittestable #13840
• Remove IndexService dep. from IndexShard #13797
• Remove ES internal deletion policies in favour of Lucenes implementations #13794
• Move ShardTermVectorService to be on indices level as TermVectorService #13786
• Move ShardPercolateService creation into IndexShard #13777
• Remove ExpressionScriptCompilationException and ExpressionScriptExecutionException #13742
• Reduced the number of ClusterStateUpdateTask variants #13735
• Add a BaseParser helper for stream parsing #13615
• Remove and forbid use of com.google.common.primitives.Ints #13596 (issue: #13224)
• Remove and forbid use of com.google.common.math.LongMath #13575 (issue: #13224)
• Remove and forbid use of com.google.common.base.Joiner #13572 (issue: #13224)
• Replace and ban next batch of Guava classes #13562 (issue: #13224)
• Remove and forbid use of com.google.common.collect.Iterables #13559 (issue: #13224)
• Replace LoadingCache usage with a simple ConcurrentHashMap #13552 (issue: #13224)
• Use Supplier instead of Reflection #13545
• Remove and forbid use of com.google.common.base.Preconditions #13540 (issue: #13224)
• Remove and forbid use of guava Function, Charsets, Collections2 #13533 (issue: #13224)
• Remove and forbid use of com.google.common.collect.ImmutableSortedMap #13525 (issue: #13224)
• Remove and forbid use of several com.google.common.util. classes #13524 (issue: #13224)
• Cleanup SearchRequest & SearchRequestBuilder #13518
• Remove and forbid use of com.google.common.collect.Queues #13498 (issue: #13224)
• Remove and forbid use of com.google.common.base.Preconditions#checkNotNull #13493 (issue: #13224)
• Remove and forbid use of com.google.common.collect.Sets #13463 (issue: #13224)
• Remove and forbid use of com.google.common.collect.Maps #13438 (issue: #13224)
• Remove use of underscore as an identifier #13353
• Remove and forbid the use of com.google.common.base.Predicate(s)? #13349 (issues: #13224, #13314)
• This commit removes com.google.common.io #13302 (issue: #13224)

Java API

• Remove copy constructors from request classes and TransportMessage type #16640 (issue: #15776)

Mapping

• Add a soft limit on the mapping depth. #17400
• Disable fielddata on text fields by defaults. #17386
• Add limit to total number of fields in mapping #17357
• Make parseMultiField part of parseField. #17313
• Automatically add a sub keyword field to string dynamic mappings. #17188
• Remove friction from the mapping changes in 5.0. #16991
• Rework norms parameters for 5.0. #16987
• Moved dynamic field handling in doc parsing to end of parsing #16798
• Remove the MapperBuilders utility class. #16609
• Make the index property a boolean. #16161
• Remove the ability to enable doc values with the fielddata.format setting. #16147
• Be stricter about parsing boolean values in mappings. #16146
• Fix default doc values to be enabled when a field is not indexed. #16141
• Dynamically map floating-point numbers as floats instead of doubles. #15319 (issue: #13851)
• Simplify MetaDataMappingService. #15217
• Remove MergeMappingException. #15177

Packaging

• Added RPM metadata #17477
• Elasticsearch ownership for data, logs, and configs #17197 (issue: #12688)
• Fail early on JDK with compiler bug #16418 (issues: #16097, #16362)
• Make security non-optional #16176
• Remove RuntimePermission("accessDeclaredMembers") #15378
• Remove Guava as a dependency #14055 (issue: #13224)
• Remove Guava as a dependency #14054 (issue: #13224)

Percolator

• Add scoring support to the percolator query #17385 (issue: #13827)
• Add query extract support for the blended term query and the common terms query #17347
• Add support for several span queries in ExtractQueryTermsService #17323
• Add support for TermsQuery in ExtractQueryTermsService #17316
• Replace percolate APIs with a percolator query #16349 (issues: #10741, #11264, #13176, #13978, #4317, #7297)

Plugin Analysis Kuromoji

• Add nbest options and NumberFilter #17173

Plugin Discovery EC2

• Add support for proxy authentication for s3 and ec2 #15293 (issue: #15268)

Plugin Ingest Attachment

• Minor attachment processor improvements #16574

Plugin Lang Painless

• Painless Clean Up #17428
• Make Painless a Module #16755
• Minor Clean up #16457
• Remove Extra String Concat Token #16382

Plugin Mapper Attachment

• minor attachments cleanups: IDE test support and EPUB format #14626

Plugin Repository Azure

• Support global repositories.azure. settings #15141 (issue: #13776)
• Add timeout settings (default to 5 minutes) #15080 (issue: #14277)
• Remove AbstractLegacyBlobContainer #14650 (issue: #13434)

Plugin Repository HDFS

• merge current hdfs improvements to master #15588

Plugin Repository S3

• Check that S3 setting buffer_size is always lower than chunk_size #17274 (issue: #17244)

Plugins

• PluginManager: Add xpack as official plugin #17227
• CliTool: Cleanup and document Terminal #16443
• Plugin cli: Improve maven coordinates detection #16384 (issue: #16376)
• Enforce plugin zip does not contain zip entries outside of the plugin dir #16361
• CliTool: Allow unexpected exceptions to propagate #16359
• Reduce complexity of plugin cli #16336
• Remove Plugin.onIndexService. #15029 (issue: #14896)
• Open up QueryCache and SearcherWrapper extension points #14303

Query DSL

• An exists query on an object should query a single term. #17186 (issue: #17131)
• Function Score Query: make parsing stricter #16617 (issue: #16583)
• Parsers should throw exception on unknown objects #14255 (issue: #10974)
• UNICODE_CHARACTER_CLASS fix #11598 (issue: #10146)

Query Refactoring

• Add infrastructure to rewrite query builders #16599
• Switch geo validation to enum #13672 (issue: #13608)

REST

• More robust handling of CORS HTTP Access Control #16092
• Add option to exclude based on paths in XContent #16017

Recovery

• Recover broken IndexMetaData as closed #17187
• Relocation source should be marked as relocating before starting recovery to primary relocation target #16500
• Operation counter for IndexShard #15956 (issue: #15900)
• Primary relocation handoff #15900 (issue: #15532)
• Remove recovery threadpools and throttle outgoing recoveries on the master #15372
• Refactor StoreRecoveryService to be a simple package private util class #13766

Reindex API

• Make reindex throttling dynamic #17262
• Throttling support for reindex #17039
• Add ingest pipeline support to reindex #16932

Scripting

• Skipping hidden files compilation for script service #16286 (issue: #15269)
• Rename Plan A to Painless #16245
• Add plumbing for script compile-time parameters #15464
• Factor mustache → modules/lang-mustache #15328

Search

• Add a soft limit on the number of shards that can be queried in a single search request. #17396
• Type filters should not have a performance impact when there is a single type. #17350
• Store _all payloads on 1 byte instead of 4. #16899
• Refuse to load fields from _source when using the fields option and support wildcards. #15017 (issues: #10783, #14489)
• Add response into ClearScrollResponse #13835 (issue: #13817)
• Shuffle shards for _only_nodes + support multiple specifications like cluster API #12575 (issues: #12546, #12700)

Search Refactoring

• Remove RescoreParseElement #17441
• Remove HighlighterParseElement #17303
• Move top level parsing of sort element to SortBuilder #17248
• Switch to using refactored SortBuilder instead of using BytesReference in serialization #17205 (issues: #17146, #17257)
• Add build() method to SortBuilder implementations #17146 (issue: #10217)
• Refactoring of Suggestions #17096 (issue: #10217)
• Move sort order field up into SortBuilder #17035
• Moves SortParser:parse(…) to only require QueryShardContext #16999 (issue: #15178)
• Change internal representation of suggesters #16873
• Make GeoDistanceSortBuilder serializable, 2nd try #16572 (issues: #15178, #16151)
• Move missing() from SortBuilder interface to class #16225 (issues: #15178, #16151)
• Remove deprecated parameters from ScriptSortBuilder #16153 (issue: #15178)
• Refactor GeoSortBuilder #16151 (issue: #15178)
• Refactor FieldSortBuilder #16127 (issue: #15178)
• Make sort order enum writable. #16124 (issue: #15178)
• Make DistanceUnit writable. #16122 (issue: #15178)
• RescoreBuilder: Add parsing and creating of RescoreSearchContext #16014 (issue: #15559)
• Make RescoreBuilder and nested QueryRescorer Writable #15953 (issue: #15559)
• Explain api: move query parsing to the coordinating node #14270
• Switch query parsers to use ParseField #14249 (issue: #8964)
• Refactoring of Aggregations #14136

Settings

• Add guard against null-valued settings #17310 (issue: #17292)
• Useful error message for null property placeholder #17293 (issue: #17292)
• Archive cluster level settings if unknown or broken #17246
• Improve error message if setting is not found #17230
• Improve upgrade experience of node level index settings #17223 (issue: #17187)
• Settings with complex matchers should not overlap #16754
• Moves GCE settings to the new infra #16722 (issue: #16720)
• Add filtering support within Setting class #16629 (issue: #16598)
• Migrate AWS settings to new settings infrastructure #16602 (issue: #16293)
• Remove gateway.initial_meta and always rely on min master nodes #16446
• Rewrite SettingsFilter to be immutable #16425
• Simplify azure settings #16363
• Convert PageCacheRecycler settings #16341
• Monitor settings #16313
• Cut over tribe node settings to new settings infra #16311
• Convert multcast plugin settings to the new infra #16295
• Convert request.headers.* to the new settings infra #16292
• Migrate Azure settings to new settings infrastructure #16291
• Validate logger settings and allow them to be reset via API #16289
• Switch NodeEnvironment’s settings to new settings #16273
• Simplify AutoCreateIndex and add more tests #16270
• Convert several pending settings #16269
• Migrate query caching settings to the new settings infra. #16267
• Convert action.auto_create_index and action.master.force_local to the new settings infra #16263
• Convert cluster.routing.allocation.type and processors to the new settings infra. #16238
• Validate tribe node settings on startup #16237
• Move node.client, node.data, node.master, node.local and node.mode to new settings infra #16230
• Moved http settings to the new settings infrastructure #16188
• Migrate network service to the new infra #16187
• Convert client.transport settings to new infra #16183
• Move discovery.* settings to new Setting infrastructure #16182
• Change over to o.e.common.settings.Setting for http settings #16181
• Convert "path.*" and "pidfile" to new settings infra #16180
• Migrate repository settings to the new settings API #16178
• Convert "indices.*" settings to new infra. #16177
• Migrate gateway settings to the new settings API. #16175
• Convert several node and test level settings #16172
• Run Metadata upgrade tool on every version #16168
• Check for invalid index settings on metadata upgrade #16156
• Validate the settings key if it’s simple chars separated by . #16120
• Validate known global settings on startup #16091
• Cut over all index scope settings to the new setting infrastrucuture #16054 (issues: #12790, #12854, #16032, #6732)
• Remove updatability of index.flush_on_close #15964 (issue: #15955)
• Move all dynamic settings and their config classes to the index level #15955 (issue: #6732)
• Always require units for bytes and time settings #15948 (issue: #11437)
• Make MetaData parsing less lenient. #15828
• Move async translog sync logic into IndexService #15584
• Remove index.merge.scheduler.notify_on_failure and default to true #15572 (issue: #15570)
• Remove cache concurrency level settings that no longer apply #14210 (issues: #13224, #13717, #7836)

Similarities

• Defining a global default similarity #16682 (issue: #16594)

Stats

• Normalize unavailable load average #16061 (issues: #12049, #14741, #15907, #15932, #15934)
• Add load averages to OS stats on FreeBSD #15934 (issue: #15917)
• Expose pending cluster state queue size in node stats #14040 (issue: #13610)

Store

• Remove support for legacy checksums #16931
• Rename index folder to index_uuid #16442 (issues: #13264, #13265, #14512, #14932, #15853)

Task Manager

• Add ability to group tasks by common parent #17341
• Add start time and duration to tasks #16829
• Combine node name and task id into single string task id #16744
• Add task status #16356 (issue: #16344)
• Extend tracking of parent tasks to master node, replication and broadcast actions #15931

Translog

• Remove ChannelReference and simplify Views #15898
• Simplify TranslogWriter to always write to a stream #15771
• Remove TranslogService and fold it into synchronous IndexShard API #13707

Bug fixes

Aggregations

• Fixes the defaults for keyed in the percentiles aggregations #17217
• Correct typo in class name of StatsAggregator #15264 (issue: #14730)

Allocation

• Replica shards must be failed before primary shards #15686

CRUD

• Prevent TransportReplicationAction to route request based on stale local routing table #16274 (issues: #12573, #12574)
• Resolves the conflict between alias routing and parent routing by applying the alias routing and ignoring the parent routing. #15371 (issue: #3068)

Cluster

• Shard state action channel exceptions #16057 (issue: #15748)

Core

• Handle RejectedExecution gracefully in TransportService during shutdown #16965

Geo

• Fix multi-field support for GeoPoint types #15702 (issue: #15701)
• Enforce distance in distance query is > 0 [ISSUE] #15135

Ingest

• Handle regex parsing errors in Gsub and Grok Processors #17260
• add on_failure exception metadata to ingest document for verbose simulate #16562
• The IngestDocument copy constructor should make a deep copy #16248 (issue: #16246)

Internal

• Enable unmap hack for java 9 #16986 (issue: #1)
• Fix issues with failed cache loads #14315
• Allow parser to move on the START_OBJECT token when parsing search source #14145
• Ensure searcher is release if wrapping fails #14107
• Avoid deadlocks in Cache#computeIfAbsent #14091 (issue: #14090)

Java API

• Fix potential NPE in SearchSourceBuilder #16905 (issue: #16902)

Mapping

• Make dynamic template parsing less lenient. #17249
• Fix dynamic mapper when its parent already has an update #17065
• Fix copy_to when the target is a dynamic object field. #15216 (issues: #111237, #11237)
• Preserve existing mappings on batch mapping updates #15130 (issues: #14899, #15129)

Packaging

• Set MAX_OPEN_FILES to 65536 #17431 (issue: #17430)
• [windows] Service command still had positional start command #17391
• Do not pass double-dash arguments on startup #17087 (issue: #17084)

Percolator

• Let PercolatorQuery’s explain use the two phase iterator #17315 (issue: #17314)

Plugin Store SMB

• Fix calling ensureOpen() on the wrong directory (master forwardport) #16395 (issue: #16383)

Plugins

• CliTool: Messages printed in Terminal should have percent char escaped #16367

Query DSL

• Resolve string dates and date math to millis before evaluating for rewrite in range query #17239
• constant_score query should throw error on more than one filter #17135 (issue: #17126)
• Single IPv4 addresses in IP field term queries #16068 (issue: #16058)
• Make strategy optional in GeoShapeQueryBuilder readFrom and writeTo #13963

Query Refactoring

• Query refactoring: set has_parent & has_child types context properly #13863
• Make sure equivalent geohashCellQueries are equal after toQuery called #13792

Recovery

• Invoke IndexingOperationListeners also when recovering from store or remote #17406
• Prevent interruption while store checks lucene files for consistency #16308
• Mark shard as recovering on the cluster state thread #14276 (issues: #13766, #14115)

Search

• Fix for search after #16271
• Do not be lenient when parsing CIDRs #14874 (issue: #14862)

Settings

• Register bootstrap settings #16513
• Add settings filtering to node info requests #16445
• Ban write access to system properties #14914

Task Manager

• Take filterNodeIds into consideration while sending task requests to nodes #17081

Translog

• Move translog recover outside of the engine #17422
• Mark shard active during recovery; push settings after engine finally inits #16250 (issues: #14121, #16209)

Upgrades

Core

• Upgrade to lucene-6.0.0-f0aa4fc. #17075
• upgrade to lucene 6.0.0-snapshot-bea235f #16964
• Upgrade to Jackson 2.7.1 #16801 (issue: #16294)

Ingest

• Update MaxMind geoip2 version to 2.6 #16837 (issue: #16801)

Internal

• Bump master (3.0-snapshot) to java 8 #13314

Search Templates

• Update mustache.java to version 0.9.1 #14053 (issue: #13224)

memo-release-notes-5.0.0-alpha2.md

5.0.0-alpha2 Release Notes

Also see Breaking changes in 5.0.

This is an alpha release and is intended for testing purposes only. Indices created in this version will not be compatible with Elasticsearch 5.0.0 GA. Upgrading 5.0.0-alpha2 to any other version is not supported.

Breaking changes

Analysis

• Analyze API : Rename filters/token_filters/char_filter in Analyze API in master #17843 (issue: #15189)

Cluster

• Remove validation errors from cluster health response #17773 (issue: #16979)

Indexed Scripts/Templates

• Store indexed scripts in the cluster state instead of the .scripts index #17650 (issue: #16651)

Packaging

• Add JVM options configuration file #17675 (issue: #17121)

Percolator

• Remove .percolator type in favour of percolator field type #17560

REST

• Remove camelCase support #17933 (issue: #8988)
• Remove case parameter from rest apis #17774 (issue: #8988)
• Disallow unquoted field names #15351 (issue: #9800)

Settings

• Remove action.get.realtime setting #17857 (issue: #12543)

Breaking Java changes

Settings

• Remove Settings.settingsBuilder. #17619

Deprecations

Query DSL

• Deprecate Indices query #17710 (issue: #12017)
• Deprecate mlt, in and geo_bbox query name shortcuts #17507

Query Refactoring

• Splits phrase and phrase_prefix in match query into MatchPhraseQueryBuilder and MatchPhrasePrefixQueryBuilder #17508

New features

Analysis

• Add fingerprint token filter and fingerprint analyzer #17873 (issue: #13325)

Plugin Analysis ICU

• Adding support for customizing the rule file in ICU tokenizer #13651 (issue: #13146)

Enhancements

CAT API

• Add _cat/tasks #17551
• Cat health supports ts=0 option #13508 (issue: #10109)

Cache

• Allow the query cache to be disabled. #16268 (issue: #15802)

Circuit Breakers

• Limit request size #17133 (issue: #16011)

Cluster

• Adds tombstones to cluster state for index deletions #17265 (issues: #16358, #17435)
• Enable acked indexing #17038 (issue: #7572)

Core

• Kill thread local leak #17921 (issues: #283, #630)
• Add heap size bootstrap check #17728 (issue: #17490)
• Remove hostname from NetworkAddress.format #17601 (issue: #17604)
• Bootstrapping bootstrap checks #17595 (issues: #17474, #17570)
• Add max map count check #16944

Geo

• Enhanced lat/long error handling #16833 (issue: #16137)

Index APIs

• Fail hot_threads in a better way if unsupported by JDK #15909

Ingest

• Streamline option naming for several processors #17892 (issue: #17835)

Internal

• Makes Script type writeable #17908 (issue: #17753)
• FiltersAggregatorBuilder: Don’t create new context for inner parsing #17851
• Clean up serialization on some stats #17832 (issue: #17085)
• Normalize registration for SignificanceHeuristics #17830 (issue: #17085)
• Make (read|write)NamedWriteable public #17829 (issue: #17682)
• Use try-with-resource when creating new parser instances where possible #17822
• Don’t pass XContentParser to ParseFieldRegistry#lookup #17794
• Internal: Remove threadlocal from document parser #17764
• Cut range aggregations to registerAggregation #17757 (issue: #17085)
• Remove ParseFieldMatcher from AbstractXContentParser #17756 (issue: #17417)
• Remove parser argument from methods where we already pass in a parse context #17738
• Switch SearchAfterBuilder to writeGenericValue #17735 (issue: #17085)
• Remove StreamableReader #17729 (issue: #17085)
• Cleanup nested, has_child & has_parent query builders for inner hits construction #17719 (issue: #11118)
• Make AllocationCommands NamedWriteables #17661
• Isolate StreamableReader #17656 (issue: #17085)
• Create registration methods for aggregations similar to those for queries #17653 (issues: #17085, #17389)
• Turn RestChannel into an interface #17643 (issue: #17133)
• Remove PROTOTYPEs from QueryBuilders #17632 (issue: #17085)
• Remove registerQueryParser #17608
• ParseField#getAllNamesIncludedDeprecated to not return duplicate names #17504
• Rework a query parser and improve registration #17458
• Clean up QueryParseContext and don’t hold it inside QueryRewrite/ShardContext #17417

Mapping

• Automatically upgrade analyzed strings with an analyzer to text. #17861
• Support dots in field names when mapping already exists #17759 (issue: #15951)
• Use the new points API to index numeric fields. #17746 (issues: #11513, #16751, #17007, #17700)
• Simplify AllEntries, AllField and AllFieldMapper: #17613

Network

• Limit request size #17133 (issue: #16011)

Packaging

• Remove unnecessary sleep from init script restart #17966
• Explicitly set packaging permissions #17912 (issue: #17634)
• Allow configuring Windows service name, description and user #17312
• rpm uses non-portable --system flag to useradd #14596 (issue: #14211)

Percolator

• PercolatorQueryBuilder cleanup by using MemoryIndex#fromDocument(…) helper #17669 (issue: #9386)

Plugins

• Cli: Improve output for usage errors #17938
• Cli: Add verbose output with zip url when installing plugin #17662 (issue: #17529)

Query DSL

• Add MatchNoDocsQuery, a query that matches no documents and prints the reason why in the toString method. #17780
• Adds ignore_unmapped option to geo queries #17751
• Adds ignore_unmapped option to nested and P/C queries #17748
• SimpleQueryParser should call MappedFieldType.termQuery when appropriate. #17678

REST

• Allow JSON with unquoted field names by enabling system property #17801 (issue: #17674)

Recovery

• TransportNodesListGatewayStartedShards should fall back to disk based index metadata if not found in cluster state #17663 (issue: #17630)

Reindex API

• Properly mark reindex’s child tasks as child tasks #17770

Search

• Fail query if it contains very large rescores #17917 (issue: #17522)

Settings

• Switch to registered Settings for all IndexingMemoryController settings #17778 (issue: #17442)

Stats

• Add points to SegmentStats. #17775 (issue: #16974)
• Remove FieldStats.Float. #17749
• Show configured and remaining delay for an unassigned shard. #17515 (issue: #17372)

Store

• Use mmapfs by default. #17616 (issue: #16983)

Suggesters

• Add bwc support for reading pre-5.0 completion index #17602

Task Manager

• Move parentTaskId into TransportRequest #17872
• Shorten the serialization of the empty TaskId #17870
• Expose whether a task is cancellable in the _tasks list API #17464 (issue: #17369)

Bug fixes

Aggregations

• Adds serialisation of sigma to extended_stats_bucket pipeline aggregation #17703 (issue: #17701)
• Fixes NPE when no window is specified in moving average request #17556 (issue: #17516)
• Fixes Filter and FiltersAggregation to work with empty query #17542 (issue: #17518)
• ExtendedStatsAggregator should also pass sigma to emtpy aggs. #17388 (issue: #17362)

Allocation

• Rebalancing policy shouldn’t prevent hard allocation decisions #17698 (issues: #14057, #14259)
• When considering the size of shadow replica shards, set size to 0 #17509 (issue: #17460)

Core

• Refactor UUID-generating methods out of Strings #17837 (issue: #17819)
• Node names cleanup #17723 (issue: #17718)
• NullPointerException from IndexingMemoryController when a version conflict happens during recovery #17569

Ingest

• Ingest does not close its factories #17626 (issue: #17625)

Internal

• Fix BulkItemResponse.Failure.toString #17871

Logging

• Add missing index name to search slow log. #17818 (issue: #17025)

Mapping

• Fix cross type mapping updates for boolean fields. #17882 (issue: #17879)
• Fix dynamic check to properly handle parents #17864 (issues: #17644, #17854)
• Fix array parsing to remove its context when finished parsing #17768
• Disallow fielddata loading on text fields that are not indexed. #17747
• Fail if an object is added after a field with the same name. #17568 (issue: #17567)

Packaging

• Fix exit code #17082

Plugin Discovery EC2

• Fix EC2 Discovery settings #17651 (issue: #16602)

Plugins

• Quote path to java binary #17496 (issue: #17495)

Query DSL

• Apply the default operator on analyzed wildcard in simple_query_string builder #17776
• Apply the default operator on analyzed wildcard in query_string builder: #17711 (issue: #2183)

REST

• Fixes reading of CORS pre-flight headers and methods #17523 (issue: #17483)
• index is a required url part for update by query #17503

Reindex API

• Reindex should never report negative throttled_until #17799 (issue: #17783)
• Reindex should gracefully handle when _source is disabled #17667 (issue: #17666)

Settings

• convert settings for ResourceWatcherService to new infrastructure #17948

Snapshot/Restore

• Fix the semantics for the BlobContainer interface #17878 (issues: #15579, #15580)
• On restore, selecting concrete indices can select wrong index #17715

Task Manager

• Shard level tasks in Bulk Action lose reference to their parent tasks #17743

Term Vectors

• Fix calculation of took time of term vectors request #17817 (issue: #12565)

Upgrades

Core

• Upgrade to lucene 6 release #17657

memo-release-notes-5.0.0-alpha3.md

5.0.0-alpha3 Release Notes

Also see Breaking changes in 5.0.

This is an alpha release and is intended for testing purposes only. Indices created in this version will not be compatible with Elasticsearch 5.0.0 GA. Upgrading 5.0.0-alpha3 to any other version is not supported.

Breaking changes

CAT API

• Row-centric output for _cat/fielddata #18068 (issue: #10249)

Inner Hits

• Drop top level inner hits in favour of inner hits defined in the query dsl #17816 (issue: #11118)

Internal

• Actually bound the generic thread pool #17017

Packaging

• Require /bin/bash in packaging #18259 (issue: #18251)
• Remove plugin script parsing of system properties #18207 (issue: #18140)

Plugin Delete By Query

• Remove Delete-By-Query plugin #18516 (issue: #18469)

Query DSL

• Lessen leniency of the query dsl. #18276

Scripting

• Allow only a single extension for a scripting engine #18332 (issue: #10598)
• Remove sandbox option for script settings, allow only registering a single language. #18226 (issues: #10598, #17114)

Settings

• Register indices.query.bool.max_clause_count setting #18341 (issue: #18336)
• Remove settings and system properties entanglement #18198 (issue: #18197)

Breaking Java changes

Allocation

• Move parsing of allocation commands into REST and remove support for plugins to register allocation commands #17802 (issue: #17894)

Discovery

• Introduce node handshake #15971 (issue: #9061)

Percolator

• Move the percolator from core to its own module #18511
• Remove percolator cache #18434

Scripting

• Remove LeafSearchScript.runAsFloat(): Nothing calls it. #18364

Search

• Refactor of query profile classes to make way for other profile implementations #18370 (issue: #10538)

New features

Ingest

• Add a Sort ingest processor #17999
• Add date_index_name processor #17973 (issue: #17814)

Reindex API

• Port Delete By Query to Reindex infrastructure #18329 (issue: #16883)

Snapshot/Restore

• Add Google Cloud Storage repository plugin #13578 (issue: #12880)

Stats

• Extend field stats to report searchable/aggregatable fields #17980 (issue: #17750)

Enhancements

Aggregations

• Rename AggregatorBuilder to AggregationBuilder #18377 (issue: #18367)
• Add the ability to use the breadth_first mode with nested aggregations (such as top_hits) which require access to score information. #18127 (issue: #9825)
• Make significant terms work on fields that are indexed with points. #18031

Allocation

• Limit retries of failed allocations per index #18467 (issue: #18417)
• Immutable ShardRouting #17821
• Add the shard’s store status to the explain API #17689 (issue: #17372)

Analysis

• Core: better error message when analyzer created without tokenizer or… #18455 (issue: #15492)
• Move AsciiFolding earlier in FingerprintAnalyzer filter chain #18281 (issue: #18266)

CAT API

• Add node name to Cat Recovery #18187 (issue: #8041)
• Add support for documented byte/size units and for micros as a time unit in _cat API #17779

Core

• Log OS and JVM on startup #18557
• Add GC overhead logging #18419
• Refactor JvmGcMonitorService for testing #18378
• Default to server VM and add client VM check #18155
• Add system bootstrap checks escape hatch #18088
• Avoid sliced locked contention in internal engine #18060 (issue: #18053)

Dates

• Support full range of Java Long for epoch DateTime #18509 (issue: #17936)

Discovery

• Log warning if minimum_master_nodes set to less than quorum #15625

Exceptions

• Make the index-too-old exception more explicit #18438
• Add index name in IndexAlreadyExistsException default message #18274

Expressions

• Support geo_point fields in lucene expressions #18096
• Add support for .empty to expressions, and some docs improvements #18077

Ingest

• Expose underlying processor to blame for thrown exception within CompoundProcessor #18342 (issue: #17823)
• Avoid string concatentation in IngestDocument.FieldPath #18108
• add ability to specify multiple grok patterns #18074 (issue: #17903)
• add ability to disable ability to override values of existing fields in set processor #17902 (issue: #17659)

Inner Hits

• Change scriptFields member in InnerHitBuilder to set #18092 (issue: #5831)

Internal

• Do not automatically close XContent objects/arrays #18549 (issue: #18433)
• Remove use of a Fields class in snapshot responses #18497
• Removes multiple toXContent entry points for SnapshotInfo #18494
• Removes unused methods in the o/e/common/Strings class #18346
• Determine content length eagerly in HttpServer #18203
• Consolidate query generation in QueryShardContext #18129
• Make reset in QueryShardContext private #18113
• Remove Strings#splitStringToArray #18110
• Add toString() to GetResponse #18102
• ConstructingObjectParser adapts ObjectParser for ctor args #17596 (issue: #17352)

Java API

• Improve adding clauses to span_near and span_or query #18485 (issue: #18478)
• QueryBuilder does not need generics. #18133

Mapping

• Adds a methods to find (and dynamically create) the mappers for the parents of a field with dots in the field name #18106 (issue: #15951)

Network

• Netty request/response tracer should wait for send #18500
• Exclude specific transport actions from request size limit check #17951

Packaging

• Don’t mkdir directly in deb init script #18503 (issue: #18307)
• Increase default heap size to 2g #18311 (issues: #16334, #17686, #18309)
• Switch init.d scripts to use bash #18308 (issue: #18259)
• Switch scripts to use bash #18251 (issue: #14002)
• Further simplifications of plugin script #18239 (issue: #18207)
• Pass ES_JAVA_OPTS to JVM for plugins script #18140 (issue: #16790)

Parent/Child

• Allow adding additional child types that point to an existing parent type #18446 (issue: #17956)

Plugin Lang Painless

• improve painless whitelist coverage of java api #18533
• Definition cleanup #18463
• Made def variable casting consistent with invokedynamic rules #18425
• Use Java 9 Indy String Concats, if available #18400 (issue: #18398)
• Add method overloading based on arity #18385
• Refactor WriterUtils to extend ASM GeneratorAdapter #18382
• Whitelist expansion #18372
• Remove boxing when loading and storing values in "def" fields/arrays, remove boxing onsimple method calls of "def" methods #18359
• Some cleanups #18352
• Use isAssignableFrom instead of relying on ClassCastException #18350
• Build descriptor of array and field load/store in code #18338
• Rename the dynamic call site factory to DefBootstrap #18335
• Cleanup of DynamicCallSite #18323
• Improve exception stacktraces #18319
• Make Line Number Available in Painless #18298
• Remove input, support params instead #18287
• Decouple ANTLR AST from Painless #18286
• _value support in painess? #18284
• Long priority over Float #18282
• _score as double, not float #18277
• Add ctx keyword to painless. #18264
• Painless doc access #18262
• Retrieve _score directly from Scorer #18258
• Implement needsScore() correctly. #18247
• Add synthetic length property as alias to Lists, so they can be used like arrays #18241
• Use better typing for dynamic method calls #18234
• Array load/store and length with invokedynamic #18232 (issue: #18201)
• Switch painless dynamic calls to invokedynamic, remove perf hack/cheat #18201
• Add fielddata accessors (.value/.values/.distance()/etc) #18169
• painless: optimize/simplify dynamic field and method access #18151
• Painless: Single-Quoted Strings #18150

Plugins

• Add plugin information for Verbose mode #18051 (issue: #16375)

Query DSL

• Enforce MatchQueryBuilder#maxExpansions() to be strictly positive #18464
• Don’t allow fuzziness for multi_match types cross_fields, phrase and phrase_prefix #18322 (issues: #6866, #7764)

REST

• CORS handling triggered whether User-Agent is a browser or not #18283
• Add semicolon query string parameter delimiter #18186 (issue: #18175)
• Enable HTTP compression by default with compression level 3 #18066 (issue: #7309)

Reindex API

• Make Reindex cancellation tests more uniform #18498
• Makes DeleteByQueryRequest implements IndicesRequest #18466
• Switch default batch size for reindex to 1000 #18340
• Teach reindex to retry on search failures #18331 (issue: #18059)
• Remove ReindexResponse in favor of BulkIndexByScrollResponse #18205
• Stricter validation of Reindex’s requests_per_second #18028

Search

• Introduces GeoValidationMethod to GeoDistanceSortBuilder #18036
• Switches from empty boolean query to matchNoDocs #18007 (issue: #17981)
• Allow binary sort values. #17959 (issue: #6077)

Search Refactoring

• Removes the now obsolete SearchParseElement implementations #18233

Snapshot/Restore

• Change BlobPath.buildAsString() method #18461
• Remove the Snapshot class in favor of using SnapshotInfo #18167 (issue: #18156)

Stats

• Do not return fieldstats information for fields that exist in the mapping but not in the index. #18212 (issue: #17980)
• Add whether the shard state fetch is pending to the allocation explain API #18119 (issue: #17372)
• Add Failure Details to every NodesResponse #17964 (issue: #3740)
• Add I/O statistics on Linux #15915 (issue: #15296)

Translog

• FSync translog outside of the writers global lock #18360

Bug fixes

Aggregations

• Fix TimeUnitRounding for hour, minute and second units #18415 (issue: #18326)
• Aggregations fix: support include/exclude strings for IP and dates #18408 (issue: #17705)
• Fix xcontent rendering of ip terms aggs. #18003 (issue: #17971)
• Improving parsing of sigma param for Extended Stats Bucket Aggregation #17562 (issue: #17499)

Analysis

• Add Character.MODIFIER_SYMBOL to the list of symbol categories. #18402 (issue: #18388)

Bulk

• Add not-null precondition check in BulkRequest #18347 (issue: #12038)

CAT API

• Expand wildcards to closed indices in /_cat/indices #18545 (issues: #16419, #17395)

Circuit Breakers

• Free bytes reserved on request breaker #18204 (issue: #18144)

Cluster

• Dangling indices are not imported if a tombstone for the index exists #18250 (issue: #18249)
• Fix issue with tombstones matching active indices in cluster state #18058 (issue: #18054)

Core

• Fix concurrency bug in IMC that could cause it to check too infrequently #18357
• Iterables.flatten should not pre-cache the first iterator #18355 (issue: #18353)
• Avoid race while retiring executors #18333
• Don’t try to compute completion stats on a reader after we already closed it #18094

Highlighting

• Skip all geo point queries in plain highlighter #18495 (issue: #17537)
• Exclude all but string fields from highlighting if wildcards are used… #18183 (issue: #17537)

Ingest

• Pipeline Stats: Fix concurrent modification exception #18177 (issue: #18126)
• Validate properties values according to database type #17940 (issue: #17683)

Internal

• Add XPointValues #18011 (issue: #18010)

Mapping

• Make doc_values accessible for _type #18220
• Fix and test handling of null_value. #18090 (issue: #18085)
• Fail automatic string upgrade if the value of index is not recognized. #18082 (issue: #18062)

Packaging

• Filter client/server VM options from jvm.options #18473
• Preserve config files from RPM install #18188 (issue: #18158)
• Fix typo in message for variable setup ES_MAX_MEM #18168
• Don’t run mkdir when $DATA_DIR contains a comma-separated list #17419 (issue: #16992)

Percolator

• Add support for MatchNoDocsQuery in percolator’s query terms extract service #18492

Plugin Discovery EC2

• Add TAG_SETTING to list of allowed tags for the ec2 discovery plugin. #18257

Plugin Lang Painless

• Remove Grammar Ambiguities #18531
• Remove if/else ANTLR ambiguity. #18428
• Fix insanely slow compilation #18410 (issue: #18398)
• Fix Bug in Painless Assignment #18379
• Fix bracket shortcuts #18263

Plugin Repository Azure

• Fix azure files removal #18451 (issues: #16472, #18436)

REST

• Do not decode path when sending error #18477 (issue: #18476)
• CORS should permit same origin requests #18278 (issue: #18256)

Search

• Fix _only_nodes preferences #18483 (issues: #12546, #12700)
• Speed up named queries. #18470
• Fix parsing single rescore element in SearchSourceBuilder #18440 (issue: #18439)
• Fail queries on not indexed fields. #18014

Settings

• Use object equality to compare versions in IndexSettings #18103
• fix exists method for list settings when using numbered setting format #17949

Snapshot/Restore

• Fix race condition in snapshot initialization #18426 (issue: #18121)
• Handle indices=["_all"] when restoring a snapshot #18025

Stats

• Add missing builder.endObject() in FsInfo #18443 (issues: #15915, #18433)

Translog

• Snapshotting and sync could cause a dead lock TranslogWriter #18481 (issues: #1, #18360, #2)

memo-release-notes-5.0.0-alpha4.md

5.0.0-alpha4 Release Notes

Also see Breaking changes in 5.0.

This is an alpha release and is intended for testing purposes only. Indices created in this version will not be compatible with Elasticsearch 5.0.0 GA. Upgrading 5.0.0-alpha4 to any other version is not supported.

Breaking changes

Aggregations

• Remove size 0 options in aggregations #18854 (issue: #18838)

CRUD

• Wait for changes to be visible by search #17986 (issue: #1063)

Core

• Remove cluster name from data path #18554 (issue: #17810)

Inner Hits

• Also do not serialize _index key in search response for parent/child inner hits #19011
• Don’t include _id, _type and _index keys in search response for inner hits #18995 (issue: #18091)
• Nested inner hits shouldn’t use relative paths #18567 (issue: #16653)

Mapping

• Remove _timestamp and _ttl on 5.x indices. #18980 (issue: #18280)

Packaging

• Remove allow running as root #18694 (issue: #18688)

Search

• Remove only node preference #18875 (issue: #18822)
• Add search preference to prefer multiple nodes #18872 (issue: #18822)

Settings

• Rename boostrap.mlockall to bootstrap.memory_lock #18669

Snapshot/Restore

• Change the default of include_global_state from true to false for snapshot restores #18773 (issue: #18569)

Breaking Java changes

Core

• Register thread pool settings #18674 (issues: #18613, #9216)

Highlighting

• Register Highlighter instances instead of classes #18859

Internal

• Cleanup ClusterService dependencies and detached from Guice #18941
• Simplify SubFetchPhase interface #18881
• Simplify FetchSubPhase registration and detach it from Guice #18862

Java API

• Remove setRefresh #18752 (issue: #1063)

Plugins

• Fail to start if plugin tries broken onModule #19025
• Simplify ScriptModule and script registration #18903
• Cut over settings registration to a pull model #18890
• Plugins cleanup #18594 (issue: #18588)

Scripting

• Move search template to lang-mustache module #18765 (issue: #17906)

New features

Aggregations

• Adds aggregation profiling to the profile API #18414 (issue: #10538)
• New Matrix Stats Aggregation module #18300 (issue: #16826)

Index APIs

• Add rollover API to switch index aliases given some predicates #18732 (issue: #18647)

Java REST Client

• Low level Rest Client #18735 (issue: #7743)

Mapping

• Expose half-floats. #18887

Scroll

• Add the ability to partition a scroll in multiple slices. #18237 (issue: #13494)

Store

• Expose MMapDirectory.preLoad(). #18880
• Add primitive to shrink an index into a single shard #18270

Enhancements

Aggregations

• Automatically set the collection mode to breadth_first in the terms aggregation when the cardinality of the field is unknown or smaller than the requested size. #18779 (issue: #9825)
• Rename PipelineAggregatorBuilder to PipelineAggregationBuilder. #18677 (issue: #18377)
• AggregatorBuilder and PipelineAggregatorBuilder do not need generics. #18368 (issue: #18133)

Allocation

• Allow _shrink to N shards if source shards is a multiple of N #18699
• Only filter intial recovery (post API) when shrinking an index #18661
• Estimate shard size for shrinked indices #18659
• Only fail relocation target shard if failing source shard is a primary #18574 (issue: #16144)
• Simplify delayed shard allocation #18351 (issue: #18293)

Analysis

• Add a MultiTermAwareComponent marker interface to analysis factories. #19028 (issues: #18064, #9978)
• Add Flags Parameter for Char Filter #18363 (issue: #18362)

Cache

• Cache FieldStats in the request cache #18768 (issue: #18717)

Cluster

• Index creation does not cause the cluster health to go RED #18737 (issues: #9106, #9126)
• Cluster Health class improvements #18673

Core

• Read Elasticsearch manifest via URL #18999 (issue: #18996)
• Throw if the local node is not set #18963 (issue: #18962)
• Improve performance of applyDeletedShards #18788 (issue: #18776)
• Bootstrap check for OnOutOfMemoryError and seccomp #18756 (issue: #18736)

Dates

• Improve TimeZoneRoundingTests error messages #18895
• Improve TimeUnitRounding for edge cases and DST transitions #18589

Expressions

• improve date api for expressions/painless fields #18658

Index APIs

• Add Shrink request source parser to parse create index request body #18802

Index Templates

• Parse and validate mappings on index template creation #8802 (issue: #2415)

Ingest

• Add ignore_failure option to all ingest processors #18650 (issue: #18493)
• new ScriptProcessor for Ingest #18193

Internal

• Hot methods redux #19016 (issue: #16725)
• Remove forked joda time BaseDateTime class #18953
• Support optional ctor args in ConstructingObjectParser #18725
• Remove thread pool from page cache recycler #18664 (issue: #18613)

Java API

• Switch QueryBuilders to new MatchPhraseQueryBuilder #18753

Logging

• Throw IllegalStateException when handshake fails due to version or cluster mismatch #18676

Mapping

• Upgrade string fields to text/keyword even if include_in_all is set. #19004 (issue: #18974)

Network

• Exclude admin / diagnostic requests from HTTP request limiting #18833 (issues: #17951, #18145)
• Do not start scheduled pings until transport start #18702

Packaging

• Remove explicit parallel new GC flag #18767
• Use JAVA_HOME or java.exe in PATH like the Linux scripts do #18685 (issue: #4913)

Percolator

• Add percolator query extraction support for dismax query #18845
• Improve percolate query performance by not verifying certain candidate matches #18696
• Improve percolator query term extraction #18610

Plugin Lang Painless

• Painless Initializers #19012
• Add augmentation #19003
• Infer lambda arguments/return type #18983
• Fix explicit casts and improve tests. #18958
• Add lambda captures #18954
• improve Debugger to print code even if it hits exception #18932 (issue: #1)
• Move semicolon hack into lexer #18931
• Add flag support to regexes #18927
• improve lambda syntax (allow single expression) #18924
• Remove useless dropArguments in megamorphic cache #18913
• non-capturing lambda support #18911 (issue: #18824)
• fix bugs in operators and more improvements for the dynamic case #18899
• improve unary operators and cleanup tests #18867 (issue: #18849)
• Add support for the find operator (=) and the match operator (==) #18858
• Remove casts and boxing for dynamic math #18849 (issue: #18847)
• Refactor def math #18847
• Add support for /regex/ #18842
• Array constructor references #18831
• Method references to user functions #18828
• Add } as a delimiter. #18827 (issue: #18821)
• Add Lambda Stub Node #18824
• Add capturing method references #18818 (issue: #18748)
• Add Functions to Painless #18810
• Add Method to Get New MethodWriters #18771
• Static For Each #18757
• Method reference support #18748 (issue: #18578)
• Add support for the new Java 9 MethodHandles#arrayLength() factory #18734
• Improve painless compile-time exceptions #18711 (issue: #18600)
• add java.time packages to painless whitelist #18621
• Add Function Reference Stub to Painless #18578

Plugins

• Add did-you-mean for plugin cli #18942 (issue: #18896)
• Plugins: Remove name() and description() from api #18906
• Emit nicer error message when trying to install unknown plugin #18876 (issue: #17226)

Query DSL

• Treat zero token in common terms query as MatchNoDocsQuery #18656
• Handle empty query bodies at parse time and remove EmptyQueryBuilder #17624 (issues: #17540, #17541)

REST

• Adding status field in _msearch error request bodies #18586 (issue: #18013)

Recovery

• index shard should be able to cancel check index on close. #18839 (issue: #12011)

Reindex API

• Implement ctx.op = "delete" on _update_by_query and _reindex #18614 (issue: #18043)

Scripting

• Compile each Groovy script in its own classloader #18918 (issue: #18572)
• Include script field even if it value is null #18384 (issue: #16408)

Scroll

• Add an index setting to limit the maximum number of slices allowed in a scroll request. #18782

Search

• Change default similarity to BM25 #18948 (issue: #18944)
• Add a parameter to cap the number of searches the msearch api will concurrently execute #18721

Sequence IDs

• Persist sequence number checkpoints #18949 (issue: #10708)
• Add sequence numbers to cat shards API #18772

Settings

• Improve error message if a setting is not found #18920 (issue: #18663)
• Cleanup placeholder replacement #17335

Snapshot/Restore

• Adds UUIDs to snapshots #18228 (issue: #18156)
• Clarify the semantics of the BlobContainer interface #18157 (issue: #15580)

Stats

• Add total_indexing_buffer/_in_bytes to nodes info API #18914 (issue: #18651)
• Allow FieldStatsRequest to disable cache #18900
• Remove index_writer_max_memory stat from segment stats #18651 (issues: #14121, #7440)
• Move DocStats under Engine to get more accurate numbers #18587

Task Manager

• Fetch result when wait_for_completion #18905
• Create get task API that falls back to the .tasks index #18682
• Add ability to store results for long running tasks #17928

Translog

• Beef up Translog testing with random channel exceptions #18997
• Do not replay into translog on local recovery #18547

Bug fixes

Allocation

• Fix recovery throttling to properly handle relocating non-primary shards #18701 (issue: #18640)

CAT API

• Fix merge stats rendering in RestIndicesAction #18720

CRUD

• Squash a race condition in RefreshListeners #18806

Circuit Breakers

• Never trip circuit breaker in liveness request #18627 (issue: #17951)

Cluster

• Fix block checks when no indices are specified #19047 (issue: #8105)
• Acknowledge index deletion requests based on standard cluster state acknowledgment #18602 (issues: #16442, #18558)

Core

• Throw exception if using a closed transport client #18722 (issue: #18708)

Dates

• Fix invalid rounding value for TimeIntervalRounding close to DST transitions #18800
• Fix problem with TimeIntervalRounding on DST end #18780

Expressions

• replace ScriptException with a better one #18600

Ingest

• Fix ignore_failure behavior in _simulate?verbose and more cleanup #18987

Internal

• Fix filtering of node ids for TransportNodesAction #18634 (issue: #18618)

Mapping

• Better error message when mapping configures null #18809 (issue: #18803)
• Process dynamic templates in order. #18638 (issues: #18625, #2401)

Packaging

• Remove extra bin/ directory in bin folder #18630

Plugin Lang Painless

• Fix compound assignment with string concats #18933 (issue: #18929)
• Fix horrible capture #18907 (issue: #18899)
• Fix Casting Bug #18871

Query DSL

• Make parsing of bool queries stricter #19052 (issue: #19034)

REST

• Get XContent params from request in Nodes rest actions #18860 (issue: #18794)

Reindex API

• Fix a race condition in reindex’s rethrottle #18731 (issue: #18744)

Search

• Require timeout units when parsing query body #19077 (issue: #19075)
• Close SearchContext if query rewrite failed #18727

Settings

• Register "cloud.node.auto_attributes" setting in EC2 discovery plugin #18678

Snapshot/Restore

• Better handling of an empty shard’s segments_N file #18784 (issue: #18707)

Stats

• Fix sync flush total shards statistics #18766

Translog

• Fix translog replay multiple operations same doc #18611 (issues: #18547, #18623)

Upgrades

Core

• Upgrade to Lucene 6.1.0. #18926
• Upgrade to lucene-6.1.0-snapshot-3a57bea. #18786
• Upgrade to Lucene 6.0.1. #18648 (issues: #17535, #28)

Dates

• Upgrade joda-time to 2.9.4 #18609 (issues: #14524, #18017)

Packaging

• Upgrade JNA to 4.2.2 and remove optionality #19045 (issue: #13245)

Plugin Discovery EC2

• Update aws sdk to 1.10.69 and add use_throttle_retries repository setting #17784 (issues: #538, #586, #589)

memo-release-notes-5.0.0-alpha5.md

5.0.0-alpha5 Release Notes

Also see Breaking changes in 5.0.

This is an alpha release and is intended for testing purposes only. Indices created in this version will not be compatible with Elasticsearch 5.0.0 GA. Upgrading 5.0.0-alpha5 to any other version is not supported.

Known networking bug in 5.0.0-alpha5

There is a major bug in the new Netty4 implementation in this release which affects any REST requests greater than 1024 bytes in size, and which will generate an exception similar to the following:

[WARN ][http.netty4] [wtOV9Vb] caught exception while handling client http traffic, closing connection [id: 0x1320b717, L:/0:0:0:0:0:0:0:1:9200 - R:/0:0:0:0:0:0:0:1:54732]
java.lang.UnsupportedOperationException: unsupported message type: DefaultFullHttpResponse (expected: ByteBuf, FileRegion)

This is due to incorrect handling of the Expect HTTP header, and it can be worked around in one of three ways:

• Use a client which does not add Expect headers (including the official clients).

• Pass a blank Except header, e.g.

  curl -H 'Expect:' ...
  

• Use Netty3 for the HTTP layer by passing the following setting at startup:

  ./bin/elasticsearch -Ehttp.type=netty3
  

Breaking changes

CAT API

• Improve cat thread pool API #19721 (issue: #19590)

Cluster

• Persistent Node Ids #19140 (issue: #17811)

Core

• Keep input time unit when parsing TimeValues #19102

Exceptions

• Die with dignity #19272 (issue: #19231)

Index APIs

• Removes write consistency level across replication action APIs in favor of wait_for_active_shards #19454 (issue: #18985)

Scripting

• Remove deprecated 1.x script and template syntax #19387 (issue: #13729)

Search

• Rename fields to stored_fields and add docvalue_fields #18992 (issue: #18943)

Settings

• Persistent Node Names #19456 (issue: #19140)
• Remove support for properties #19398 (issues: #19388, #19391)

Breaking Java changes

CRUD

• Removing isCreated and isFound from the Java API #19645 (issues: #19566, #19631)

Internal

• Clean up BytesReference #19196

Java API

• Add a dedicated client/transport project for transport-client #19435 (issue: #19412)

Network

• Factor out abstract TCPTransport* classes to reduce the netty footprint #19096

Plugins

• Migrate query registration from push to pull #19376
• Add components getter as bridge between guice and new plugin init world #19371
• Remove CustomNodeAttributes extension point #19348
• Add RepositoryPlugin interface for registering snapshot repositories #19324
• Simplified repository api for snapshot/restore #19292
• Switch most search extensions from push to pull #19238
• Move RestHandler registration to ActionModule and ActionPlugin #19165
• Pull actions from plugins #19108
• Switch analysis from push to pull #19073
• Remove guice from Mapper plugins #19018

Scripting

• Remove o.e.script.Template class and move template query to lang-mustache module #19425 (issue: #16314)

Settings

• Remove node.mode and node.local settings #19428

Snapshot/Restore

• Removes extra writeBlob method in BlobContainer #19727 (issue: #18528)

Deprecations

CRUD

• Deprecate found and created in delete and index rest responses #19633

Plugin Discovery Azure Classic

• Deprecate discovery-azure and rename it to discovery-azure-classic #19186 (issue: #19144)

Templates

• Deprecate template query #19607 (issue: #19390)

New features

Aggregations

• Split regular histograms from date histograms. #19551 (issues: #4847, #8082)

Circuit Breakers

• Circuit break on aggregation bucket numbers with request breaker #19394 (issue: #14046)

Ingest

• ingest-useragent plugin #19074

Java REST Client

• Introduce async performRequest method #19400

Mapping

• Add scaled_float. #19264 (issues: #15939, #1941)

Plugin Repository S3

• Add support for path_style_access #15114

Reindex API

• Reindex from remote #18585 (issue: #17447)

Translog

• Add elasticsearch-translog CLI tool with truncate command #19342 (issue: #19123)

Enhancements

Aggregations

• Make the heuristic to compute the default shard size less aggressive. #19659
• Add _bucket_count option to buckets_path #19571 (issue: #19553)
• Remove AggregationStreams #19507
• Migrate serial_diff aggregation to NamedWriteable #19483
• Migrate most remaining pipeline aggregations to NamedWriteable #19480
• Migrate moving_avg pipeline aggregation to NamedWriteable #19420
• Migrate matrix_stats to NamedWriteable #19418
• Migrate derivative pipeline aggregation to NamedWriteable #19407
• Migrate top_hits, histogram, and ip_range aggregations to NamedWriteable #19375
• Migrate nested, reverse_nested, and children aggregations to NamedWriteable #19374
• Migrate geohash_grid and geo_bounds aggregations to NamedWriteable #19372
• Clean up significant terms aggregation results #19365
• Migrate range, date_range, and geo_distance aggregations to NamedWriteable #19321
• Migrate terms aggregation to NamedWriteable #19277
• Migrate sampler and missing aggregations to NamedWriteable #19259
• Migrate global, filter, and filters aggregation to NamedWriteable #19220
• Migrate the cardinality, scripted_metric, and geo_centroid aggregations to NamedWriteable #19219
• Use a static default precision for the cardinality aggregation. #19215
• Migrate more aggregations to NamedWriteable #19199
• Migrate stats and extended stats to NamedWriteable #19198
• Migrate sum, min, and max aggregations over to NamedWriteable #19194
• Start migration away from aggregation streams #19097

Analysis

• Specify custom char_filters/tokenizer/token_filters in the analyze API #15671 (issue: #8878)

CAT API

• Includes the index UUID in the _cat/indices API #19204 (issue: #19132)

CRUD

• #19664 Renaming operation to result and reworking responses #19704 (issue: #19664)
• Adding _operation field to index, update, delete response. #19566 (issues: #19267, #9642, #9736)

Cache

• Enable option to use request cache for size > 0 #19472

Cluster

• Batch process node left and node failure #19289 (issue: #19282)
• Index creation waits for write consistency shards #18985
• Inline reroute with process of node join/master election #18938 (issue: #17270)

Core

• Makes index creation more friendly #19450 (issue: #9126)
• Clearer error when handling fractional time values #19158 (issue: #19102)

Discovery

• Do not log cluster service errors at after joining a master #19705

Exceptions

• Make NotMasterException a first class citizen #19385
• Do not catch throwable #19231

Geo

• GeoBoundingBoxQueryBuilder should throw IAE when topLeft and bottomRight are the same coordinate #18668 (issues: #18458, #18631)

Ingest

• Add REST _ingest/pipeline to get all pipelines #19603 (issue: #19585)
• Show ignored errors in verbose simulate result #19404 (issue: #19319)
• update foreach processor to only support one applied processor. #19402 (issue: #19345)
• Skip the execution of an empty pipeline #19200 (issue: #19192)

Internal

• Make Priority an enum #19448
• Snapshot UUIDs in blob names #19421 (issues: #18156, #18815, #19002, #7540)
• Add RestController method for deprecating in one step #19343
• Tighten ensure atomic move cleanup #19309 (issue: #19036)
• Enable checkstyle ModifierOrder #19214
• Expose task information from NodeClient #19189
• Changed rest handler interface to take NodeClient #19170
• Deprecate ExceptionsHelper.detailedMessage #19160 (issue: #19069)
• Factor out ChannelBuffer from BytesReference #19129
• Cleanup Compressor interface #19125
• Require executor name when calling scheduleWithFixedDelay #17538

Java API

• Start from a random node number so that clients do not overload the first node configured #14143

Java REST Client

• Add response body to ResponseException error message #19653 (issue: #19653)
• Simplify Sniffer initialization and automatically create the default HostsSniffer #19599
• Remove duplicate dependency declaration for http client #19580 (issue: #19281)
• Add callback to customize http client settings #19373
• Rest Client: add short performRequest method variants without params and/or body #19340 (issue: #19312)

Logging

• Add log message about enforcing bootstrap checks #19451
• Improve logging for batched cluster state updates #19255
• Send HTTP Warning Header(s) for any Deprecation Usage from a REST request #17804 (issue: #17687)

Mapping

• Elasticsearch should reject dynamic templates with unknown match_mapping_type. #17285 (issue: #16945)

Network

• Explicitly tell Netty to not use unsafe #19786 (issues: #19562, #5624)
• Enable Netty 4 extensions #19767 (issue: #19526)
• Modularize netty #19392
• Simplify TcpTransport interface by reducing send code to a single send method #19223

Percolator

• Also support query term extract for queries wrapped inside a FunctionScoreQuery #19184
• Add support for synonym query to percolator query term extraction #19066

Plugin Discovery EC2

• Use DefaultAWSCredentialsProviderChain AWS SDK class for credentials #19561 (issue: #19556)
• Support new Asia Pacific (Mumbai) ap-south-1 AWS region #19112 (issue: #19110)

Plugin Discovery GCE

• Allow _gce_ network when not using discovery gce #15765 (issue: #15724)

Plugin Lang Painless

• Change Painless Tree Structure for Variable/Method Chains #19459
• Add replaceAll and replaceFirst #19070

Plugin Mapper Size

• Add doc values support to the _size field in the mapper-size plugin #19217 (issue: #18334)

Plugins

• Add ScriptService to dependencies available for plugin components #19770
• Log one plugin info per line #19441
• Make rest headers registration pull based #19440
• Add resource watcher to services available for plugin components #19401
• Add some basic services to createComponents for plugins #19380
• Make plugins closeable #19137
• Plugins: Add status bar on download #18695

Query DSL

• Allow empty json object in request body in _count API #19595 (issue: #19422)

REST

• Add Location header to the index, update, and create APIs #19509 (issue: #19079)
• Add support for wait_for_events to the _cluster/health REST endpoint #19432 (issue: #19419)
• Rename Search Template REST spec names #19178

Recovery

• Non-blocking primary relocation hand-off #19013 (issues: #15900, #18553)

Reindex API

• Only ask for _version we need it #19693 (issue: #19135)
• Use fewer threads when reindexing-from-remote #19636
• Support authentication with reindex-from-remote #19310
• Support requests_per_second=-1 to mean no throttling in reindex #19101 (issue: #19089)

Scripting

• Remove ClusterState from compile api #19136
• Mustache: Render Map as JSON #18856 (issue: #18970)

Search

• Limit batch size when scrolling #19367 (issue: #19249)
• Record method counts while profiling query components #18302

Settings

• Validates new dynamic settings from the current state #19122 (issue: #19046)

Snapshot/Restore

• BlobContainer#writeBlob no longer can overwrite a blob #19749 (issue: #15579)
• More resilient blob handling in snapshot repositories #19706 (issues: #18156, #18815, #19421, #7540)
• Adding repository index generational files #19002 (issue: #18156)
• Raised IOException on deleteBlob #18815 (issue: #18530)

Stats

• Add missing field type in the FieldStats response. #19241 (issue: #17750)
• Expose the ClusterInfo object in the allocation explain output #19106 (issue: #14405)

Bug fixes

Aggregations

• Undeprecates aggs in the search request #19674 (issue: #19504)
• Change how nested and reverse_nested aggs know about their nested depth level #19550 (issues: #11749, #12410)
• Make ExtendedBounds immutable #19490 (issue: #19481)
• Fix potential AssertionError with include/exclude on terms aggregations. #19252 (issue: #18575)
• Pass resolved extended bounds to unmapped histogram aggregator #19085 (issue: #19009)
• Fix "key_as_string" for date histogram and epoch_millis/epoch_second format with time zone #19043 (issue: #19038)

Allocation

• Fix NPE when initializing replica shard has no UnassignedInfo #19491 (issue: #19488)
• Make shard store fetch less dependent on the current cluster state, both on master and non data nodes #19044 (issue: #18938)

Analysis

• Fix analyzer alias processing #19506 (issue: #19163)

CAT API

• Fixes cat tasks operation in detailed mode #19759 (issue: #19755)
• Add index pattern wildcards support to _cat/shards #19655 (issue: #19634)

Cluster

• Allow routing table to be filtered by index pattern #19688
• Use executor’s describeTasks method to log task information in cluster service #19531

Core

• Makes m case sensitive in TimeValue #19649 (issue: #19619)
• Guard against negative result from FileStore.getUsableSpace when picking data path for a new shard #19554
• Handle rejected execution exception on reschedule #19505

Dates

• Make sure TimeIntervalRounding is monotonic for increasing dates #19020

Geo

• Incomplete results when using geo_distance for large distances #17578

Highlighting

• Plain highlighter should ignore parent/child queries #19616 (issue: #14999)
• Let fast vector highlighter also extract terms from the nested query’s inner query. #19337 (issue: #19265)

Index APIs

• Fixes active shard count check in the case of all shards #19760
• Add zero-padding to auto-generated rollover index name increment #19610 (issue: #19484)

Ingest

• Fix NPE when simulating a pipeline with no id #19650
• Change foreach processor to use ingest metadata for array element #19609 (issue: #19592)
• No other processors should be executed after on_failure is called #19545
• rethrow script compilation exceptions into ingest configuration exceptions #19318
• Rename from ingest-useragent plugin to ingest-user-agent and its processor from useragent to user_agent #19261

Inner Hits

• Ensure that that InnerHitBuilder uses rewritten queries #19360 (issue: #19353)

Internal

• Priority values should be unmodifiable #19447
• Extract AbstractBytesReferenceTestCase #19141

Java REST Client

• Rest Client: add slash to log line when missing between host and uri #19325 (issue: #19314)
• Rest Client: HostsSniffer to set http as default scheme #19306

Logging

• Only log running out of slots when out of slots #19637

Mapping

• Mappings: Fix detection of metadata fields in documents #19765
• Fix not_analyzed string fields to error when position_increment_gap is set #19510
• Automatically created indices should honor index.mapper.dynamic. #19478 (issue: #17592)

Network

• Verify lower level transport exceptions don’t bubble up on disconnects #19518 (issue: #19096)

Packaging

• Disable service in pre-uninstall #19328

Parent/Child

• Make sure that no _parent#null gets introduces as default _parent mapping #19470 (issue: #19389)

Plugin Discovery Azure Classic

• Make discovery-azure plugin work again #19062 (issues: #15630, #18637)

Plugin Discovery EC2

• Fix EC2 discovery settings #18690 (issues: #18652, #18662)

Plugin Discovery GCE

• Fix NPE when GCE region is empty #19176 (issue: #16967)

Plugin Repository Azure

• Register group setting for repository-azure accounts #19086

Plugin Repository S3

• Add missing permission to repository-s3 #19128 (issues: #18539, #19121)
• Fix repository S3 Settings and add more tests #18703 (issues: #18662, #18690)

Query DSL

• Throw ParsingException if a query is wrapped in an array #19750 (issue: #12887)
• Restore parameter name auto_generate_phrase_queries #19514 (issue: #19512)

REST

• Fixes CORS handling so that it uses the defaults #19522 (issue: #19520)

Recovery

• Move reset recovery into RecoveriesCollection #19466 (issue: #19473)
• Fix replica-primary inconsistencies when indexing during primary relocation with ongoing replica recoveries #19287 (issue: #19248)

Search

• Don’t recursively count children profile timings #19397 (issue: #18693)
• fix explain in function_score if no function filter matches #19185
• Fix NPEs due to disabled source #18957

Settings

• Validate settings against dynamic updaters on the master #19088 (issue: #19046)

Stats

• Fix serialization bug in allocation explain API. #19494
• Allocation explain: Also serialize includeDiskInfo field #19492

Store

• Tighten up concurrent store metadata listing and engine writes #19684 (issue: #19416)
• Make static Store access shard lock aware #19416 (issue: #18938)
• Catch assertion errors on commit and turn it into a real exception #19357 (issue: #19356)

Upgrades

Network

• Dependencies: Upgrade to netty 4.1.4 #19689
• Introduce Netty 4 #19526 (issue: #3226)
• Upgrade to netty 3.10.6.Final #19235

memo-release-notes-5.0.0-beta1.md

5.0.0-beta1 Release Notes

Also see Breaking changes in 5.0.

This is a beta release and is intended for testing purposes only. Indices created in this version will not be compatible with Elasticsearch 5.0.0 GA. Upgrading 5.0.0-beta1 to any other version is not supported.

Breaking changes

Analysis

• Remove token_filter in _analyze API #20285 (issue: #20283)
• Removes support for adding aliases to analyzers #19994 (issue: #18244)

CRUD

• Fixed naming inconsistency for fields/stored_fields in the APIs #20166 (issues: #18943, #20155)
• Disallow creating indices starting with - or + #20033 (issue: #19800)

Core

• Remove minimum master nodes bootstrap check #20082

Engine

• Optimize indexing for the autogenerated ID append-only case #20211 (issue: #19813)

Geo

• Deprecate GeoDistance enums and remove geo distance script helpers #19783

Logging

• Introduce Log4j 2 #20235 (issues: #16030, #17697)

Packaging

• Rename service.bat to elasticsearch-service.bat #20496 (issue: #17528)
• Remove -D handling in args for windows plugin script #20378 (issue: #18207)
• Set default min heap equal to default max heap #20080 (issues: #16334, #17728, #18311)

Plugin Lang Painless

• Make Painless the Default Language #20017 (issue: #19960)

REST

• Parameter improvements to Cluster Health API wait for shards #20223 (issue: #20216)
• Switch indices.exists_type from {index}/{type} to {index}/_mapping/{type}. #20055 (issue: #15613)
• Only use PUT for index creation, not POST. #20001 (issues: #15613, #18160)

Scripting

• Hardcode painless as the default scripting lang and add legacy script default for stored scripts #20310 (issue: #20122)

Settings

• Default max local storage nodes to one #19964 (issues: #19679, #19748)

Breaking Java changes

Core

• Remove ability to plug-in TransportService #20505

Internal

• Remove TransportService#registerRequestHandler leniency #20469 (issue: #20468)
• Consolidate search parser registries #20000
• Move all FetchSubPhases to o.e.search.fetch.subphase #19702
• Squash the rest of o.e.rest.action #19698

Plugins

• Cleanup sub fetch phase extension point #20382
• Remove IndexTemplateFilter #20072
• Switch custom ShardsAllocators to pull based model #20071
• Make custom allocation deciders use pull based extensions #20040

Search

• Remove FetchSubPhaseParseElement #20350

Deprecations

Geo

• Deprecate optimize_bbox on geodistance queries #20032 (issue: #20014)
• Deprecate geohash parameters for geo_point parser #20030 (issue: #20009)

Scripting

• Deprecate Groovy, Python, and Javascript #20244 (issue: #20129)

New features

Aliases

• Add an alias action to delete an index #20184 (issue: #20064)

Analysis

• Exposing lucene 6.x minhash filter. #20206 (issue: #20149)

Geo

• Cut over geo_point field and queries to new LatLonPoint type #20315 (issue: #20314)

Plugin Discovery File

• File-based discovery plugin #20394 (issue: #20323)

Search

• Add the ability to disable the retrieval of the stored fields entirely #20026

Suggesters

• Add support for returning documents with completion suggester #19536 (issue: #10746)

Enhancements

Allocation

• Change DiskThresholdDecider’s behavior when factoring in leaving shards #20349
• Verify AllocationIDs in replication actions #20320
• Mark shard as stale on non-replicated write, not on node shutdown #20023 (issue: #18919)
• Add routing changes API to RoutingAllocation #19992
• Primary shard allocator observes limits in forcing allocation #19811 (issue: #19446)
• Use primary terms as authority to fail shards #19715
• Add recovery source to ShardRouting #19516

Analysis

• Stop using cached component in _analyze API #19929 (issue: #19827)

Benchmark

• Add client-benchmark-noop-api-plugin to stress clients even more in benchmarks #20103

CAT API

• Add health status parameter to cat indices API #20393

Circuit Breakers

• Circuit break the number of inline scripts compiled per minute #19694 (issue: #19396)

Cluster

• Add clusterUUID to RestMainAction output #20503

Core

• Add serial collector bootstrap check #20558
• Rename Netty TCP transports thread factories from http_* to transport_* #20207
• Do not log full bootstrap checks exception #19989
• Mark halting the virtual machine as privileged #19923 (issues: #19272, #19806)

Engine

• Only try to read new segments info if we really flushed the index #20474
• Use _refresh instead of reading from Translog in the RT GET case #20102

Exceptions

• Improve startup exception #20083

Index Templates

• Add "version" field to Templates #20353 (issue: #20171)

Ingest

• Add "version" field to Pipelines #20343 (issue: #20171)
• Make it possible for Ingest Processors to access AnalysisRegistry #20233
• add ignore_missing option to convert,trim,lowercase,uppercase,grok,rename #20194 (issue: #19995)
• Add support for parameters to the script ingest processor #20136
• introduce the JSON Processor #20128 (issue: #20052)
• Allow rename processor to turn leaf fields into branch fields #20053 (issue: #19892)
• remove ability to set field value in script-processor configuration #19981

Internal

• Remove poor-mans compression in InternalSearchHit and friends #20472
• Don’t register SearchTransportService handlers more than once #20468
• Unguice SearchModule #20456
• Deguice SearchService and friends #20423
• NodeStats classes to implement Writeable rather then Streamable #20327
• More info classes to implement Writeable rather than Streamable #20288 (issue: #20255)
• Internal: Split disk threshold monitoring from decider #20018
• Switching LockObtainFailedException over to ShardLockObtainFailedException #19991 (issue: #19978)
• update and delete by query requests to implement IndicesRequest.Replaceable #19961
• VersionFetchSubPhase should not use Versions#loadDocIdAndVersion #19944

Java API

• Ensure PutMappingRequest.buildFromSimplifiedDef input are pairs #19837 (issue: #19836)

Java REST Client

• Add support for a RestClient path prefix #20190
• Add "Async" to the end of each Async RestClient method #20172 (issue: #20168)
• Allow RestClient to send array-based headers #20151

Logging

• Ensure logging is initialized in CLI tools #20575
• Give useful error message if log config is missing #20493
• Complete Elasticsearch logger names #20457 (issue: #20326)
• Add node name to decider trace logging #20437 (issue: #20379)
• Logging shutdown hack #20389 (issue: #20304)
• Disable console logging #20387
• Skip loading of jansi from log4j2 #20334
• Configure AWS SDK logging configuration #20313 (issue: #20294)
• Warn if unsupported logging configuration present #20309
• Size limit deprecation logs #20287 (issues: #20235, #20254)
• Increase visibility of deprecation logger #20254 (issue: #11033)
• Add shardId and node name in search slow log #19949 (issue: #19735)

Mapping

• Automatically downgrade text and keyword to string on indexes imported from 2.x #20177
• Do not parse numbers as both strings and numbers when not included in _all. #20167
• Don’t index the _version field #20132
• The root object mapper should support updating numeric_detection, date_detection and dynamic_date_formats. #20119 (issue: #20111)
• Automatically upgrade analyzed string fields that have index_options or position_increment_gap set. #20002 (issue: #19974)
• Mappings: Support dots in field names in mapping parsing #19899 (issue: #19443)
• Save one utf8 conversion in KeywordFieldMapper. #19867
• Do not parse the created version from the settings every time a field is parsed. #19824

Network

• Avoid early initializing Netty #19819 (issue: #5644)
• Network: Allow to listen on virtual interfaces. #19568 (issues: #17473, #19537)

Packaging

• Add quiet option to disable console logging #20422 (issues: #15315, #16159, #17220)
• Explicitly disable Netty key set replacement #20249

Plugin Lang Painless

• Disable regexes by default in painless #20427 (issue: #20397)
• Catch OutOfMemory and StackOverflow errors in Painless #19936

Plugin Repository S3

• Extract AWS Key from KeyChain instead of using potential null value #19560 (issues: #18703, #19557)

Plugins

• Print message when removing plugin with config #20338
• Plugins: Update official plugin location with unified release #19996
• Allow plugins to upgrade global custom metadata on startup #19962
• Switch aggregations from push to pull #19839
• Display plugins versions #18683

Query DSL

• Throw exception when multiple field names are provided as part of query short syntax #19871 (issue: #19791)
• Query parsers to throw exception when multiple field names are provided #19791 (issue: #19547)

REST

• Add exclusion support to response filtering #19865

Recovery

• Pass on maxUnsafeAutoIdTimestamp on recovery / relocation #20300

Scripting

• Parse script on storage instead of on retrieval #20356
• Migrate elasticsearch native script examples to the main repo #19334 (issue: #14662)

Settings

• Make action.auto_create_index setting a dynamic cluster setting #20274 (issue: #7513)
• Removes space between # and the setting in elasticsearch.yml #20094 (issue: #20090)

Snapshot/Restore

• Delete differing files in the store before restoring #20220 (issue: #20148)
• Adds ignoreUnavailable option to the snapshot status API #20066 (issue: #18522)
• Check restores in progress before deleting a snapshot #19853
• Snapshot repository cleans up empty index folders #19751

Stats

• Add mem section back to cluster stats #20255 (issue: #17278)
• Add network types to cluster stats #20144

Suggesters

• Move SuggestUtils methods to their respective caller classes #19914 (issue: #19906)
• Remove payload option from completion suggester #19877 (issue: #19536)

Task Manager

• Rename Task Persistence into Storing Task Results #19982

Translog

• Fsync documents in an async fashion #20145
• Add checksumming and versions to the Translog’s Checkpoint files #19797

Bug fixes

Aggregations

• Fix agg profiling when using breadth_first collect mode #20156
• Throw exception when maxBounds greater than minBounds #19855 (issue: #19833)

Allocation

• Upon being elected as master, prefer joins' node info to existing cluster state #19743

Analysis

• Can load non-PreBuiltTokenFilter in Analyze API #20396

CRUD

• GET operations should not extract fields from _source. #20158 (issues: #15017, #20102)

Cache

• Serialize index boost and phrase suggest collation keys in a consistent order #20081 (issue: #19986)

Cluster

• Fixes issue with dangling index being deleted instead of re-imported #19666

Discovery

• Update discovery nodes after cluster state is published #20409
• Add current cluster state version to zen pings and use them in master election #20384 (issue: #20348)

Engine

• Take refresh IOExceptions into account when catching ACE in InternalEngine #20546 (issue: #19975)
• Don’t suppress AlreadyClosedException #19975 (issue: #19861)

Highlighting

• Fix highlighting of MultiTermQuery within a FunctionScoreQuery #20400 (issue: #20392)
• Enable BoostingQuery with FVH highlighter #19984 (issue: #19985)

Index APIs

• Fix IndexNotFoundException in multi index search request. #20188 (issue: #3839)

Ingest

• Don’t rebuild pipeline on every cluster state update #20189
• Add dotexpander processor #20078

Internal

• Ensure elasticsearch doesn’t start with unuspported indices #20514 (issue: #20512)
• Remove ListTasksResponse#setDiscoveryNodes() #19773 (issue: #19772)

Java API

• fix IndexResponse#toString to print out shards info #20562
• Add NamedWriteables from plugins to TransportClient #19825 (issue: #19764)

Logging

• Avoid unnecessary creation of prefix loggers #20571 (issue: #20570)
• Fix logging hierarchy configs #20463
• Fix prefix logging #20429
• Hack around Log4j bug rendering exceptions #20306 (issue: #20304)
• Avoid prematurely triggering logger initialization #20170

Mapping

• Mapping updates on objects should propagate include_an_all. #20051
• Validate blank field name #19860 (issue: #19251)

Network

• Fix expect 100 continue header handling #19904 (issue: #19834)
• Fixes netty4 module’s CORS config to use defaults #19874
• Fix various concurrency issues in transport #19675

Packaging

• [Packaging] Do not remove scripts directory on upgrade #20452
• [Package] Remove bin/lib/modules directories on RPM uninstall/upgrade #20448
• RESTART_ON_UPGRADE incorrectly named ES_RESTART_ON_UPGRADE in sysconfig #19976 (issue: #19950)
• Fix handling of spaces for jvm.options on Windows #19951 (issue: #19941)

Percolator

• Fail indexing percolator queries containing either a has_child or has_parent query #20229 (issue: #2960)

Plugin Ingest Attachment

• Adds content-length as number #19927 (issue: #19924)

Plugin Ingest GeoIp

• [ingest-geoip] update geoip to not include null-valued results from #20455

Plugin Lang Painless

• Fix break bug in for/foreach loops. #20146

Plugins

• Use sysprop like with es.path.home to pass conf dir #18870 (issue: #18689)

Query DSL

• Fix silently accepting malformed queries #20515 (issue: #20500)
• query_string_query should take term length into consideration when fuzziness is auto #20299 (issue: #15972)

Recovery

• Fix concurrency issues between cancelling a relocation and marking shard as relocated #20443

Reindex API

• Fix reindex with transport client #19997 (issues: #19773, #19979)
• Make reindex and lang-javascript compatible #19933

Scripting

• Fix Javascript OOM build Failure #20307
• Fix propagating the default value for script settings #20183 (issue: #20159)
• Catch and wrap AssertionError and NoClassDefFoundError in groovy scripts #19958 (issues: #19806, #19923)

Search

• Throw error when trying to fetch fields from source and source is disabled #20424 (issues: #20093, #20408)
• Source filtering should keep working when the source contains numbers greater than Long.MAX_VALUE. #20278 (issue: #11508)
• Fix NPE when running a range query on a scaled_float with no upper bound. #20253
• Add ref-counting to SearchContext to prevent accessing already closed readers #20095 (issue: #20008)
• Fix NPE during search with source filtering if the source is disabled. #20093 (issue: #7758)
• Restore assignment of time value when deserializing a scroll instance #19977 (issue: #18820)
• Fix explain output for dfs query #19972 (issue: #15369)

Settings

• Validate max thread/merge settings #20383 (issue: #20380)
• Fix filter cache setting to allow percentages #20335 (issue: #20330)
• Move cluster.routing.allocation.same_shard.host setting to new settings infrastructure #20046 (issue: #20045)

Stats

• Fix FieldStats deserialization of ip field #20522 (issue: #20516)

Translog

• Fix RAM usage estimation of LiveVersionMap. #20123 (issue: #19787)

Upgrades

Core

• Upgrade to Lucene 6.2.0 #20147 (issue: #20092)
• Update to jackson 2.8.1 #18939 (issue: #18076)

Network

• Upgrade to Netty 4.1.5 #20222 (issue: #19786)

Scripting

• Dependencies: Updates to mustache 0.9.3 #20337

memo-release-notes-5.0.0-GA.md

5.0.0 GA Release Notes

The list below cover changes between 5.0.0-rc1 and 5.0.0 GA only.

Also see Breaking changes in 5.0.

Enhancements

Circuit Breakers

• Cluster Settings Updates should not trigger circuit breakers. #20827

Plugins

• Revert "Display plugins versions" #20807 (issues: #18683, #20668)

REST

• Add a REST spec for the create API #20924

Settings

• Add precise logging on unknown or invalid settings #20951 (issue: #20946)

Bug fixes

Aggregations

• Fixed writeable name from range to geo_distance #20860

Allocation

• Keep a shadow replicas' allocation id when it is promoted to primary #20863 (issue: #20650)

Index APIs

• Only negate index expression on all indices with preceding wildcard #20898 (issues: #19800, #20033)

Internal

• Prevent AbstractArrays from release bytes more than once #20819

Mapping

• Allow position_gap_increment for fields in indices created prior to 5.0 #20806 (issues: #19510, #20413)

Network

• Fix connection close header handling #20956 (issue: #20938)
• Ensure port range is readable in the exception message #20893

Plugin Repository S3

• Fixes leading forward slash in S3 repository base_path #20861

Query DSL

• Fixes MultiMatchQuery so that it doesn’t provide a null context #20882
• Fix match_phrase_prefix query with single term on _all field #20471 (issue: #20470)

Scripting

• Add support for booleans in scripts #20950 (issue: #20949)

Search

• Do not cache script queries. #20799 (issue: #20763)

Regressions

Highlighting

• Handle SynonymQuery extraction for the FastVectorHighlighter #20829 (issue: #20781)

memo-release-notes-5.0.0-rc1.md

5.0.0-rc1 Release Notes

Also see Breaking changes in 5.0.

This is a release candidate and is intended for testing purposes only and should not be deployed in production.

Breaking changes

Core

• Remove ignore system bootstrap checks #20511

Internal

• _flush should block by default #20597 (issue: #20569)

Plugin Lang Painless

• Remove all date now methods from Painless #20766 (issue: #20762)

Enhancements

Cluster

• Skip shard management code when updating cluster state on client/tribe nodes #20731

Core

• Add production warning for pre-release builds #20674

Index APIs

• Add date-math support to _rollover #20709

Plugins

• Provide error message when plugin id is missing #20660

REST

• Add response params to REST params did you mean #20753 (issues: #20722, #20747)
• Add did you mean to strict REST params #20747 (issue: #20722)

Tribe Node

• Fail fast when executing master level write operations via a tribe node #15441 (issue: #13290)

Bug fixes

Aggregations

• Fix date_range aggregation to not cache if now is used #20740
• The top_hits aggregation should compile scripts only once. #20738

Allocation

• IndicesClusterStateService should clean local started when re-assigns an initializing shard with the same aid #20687
• IndexRoutingTable.initializeEmpty shouldn’t override supplied primary RecoverySource #20638 (issue: #20637)
• Update incoming recoveries stats when shadow replica is reinitialized #20612
• index.routing.allocation.initial_recovery limits replica allocation #20589

Analysis

• Named analyzer should close the analyzer that it wraps #20197

Cache

• Prevent requests that use scripts or now() from being cached #20750 (issue: #20645)

Core

• StoreStatsCache should also ignore AccessDeniedException when checking file size #20790 (issue: #17580)

Ingest

• no null values in ingest configuration error messages #20616
• JSON Processor was not properly added #20613

Internal

• IndicesAliasesRequest should not implement CompositeIndicesRequest #20726

Logging

• Fix logger when you can not create an azure storage client #20670 (issues: #20633, #20669)

Plugin Lang Painless

• Fix String Concatenation Bug In Painless #20623

REST

• Change separator for shards preference #20786 (issues: #20722, #20769)
• Remove lenient URL parameter parsing #20722 (issue: #14719)

Scripting

• Native scripts should be created once per index, not per segment. #20609

Search

• Make range queries round up upper bounds again. #20582 (issues: #20579, #8889)

Settings

• Fix Setting.timeValue() method #20696 (issue: #20662)
• Add a hard limit for index.number_of_shard #20682
• Include complex settings in settings requests #20622

memo-release-notes-5.0.0.md

5.0.0 Combined Release Notes

The list below covers all changes from 5.0.0-alpha1 to 5.0.0 GA excluding changes which were already released in the 2.x series, which can be found in 5.0.0-alpha1 Release Notes (Changes previously released in 2.x).

Also see Breaking changes in 5.0.

Breaking changes

Aggregations

• Remove size 0 options in aggregations #18854 (issue: #18838)

Aliases

• make get alias expand to open and closed indices by default #15954 (issue: #14982)
• Remove deprecated indices.get_aliases #13906

Allocation

• Remove DisableAllocationDecider #13313

Analysis

• Remove token_filter in _analyze API #20285 (issue: #20283)
• Removes support for adding aliases to analyzers #19994 (issue: #18244)
• Analyze API : Rename filters/token_filters/char_filter in Analyze API in master #17843 (issue: #15189)

CAT API

• Improve cat thread pool API #19721 (issue: #19590)
• Row-centric output for _cat/fielddata #18068 (issue: #10249)
• Add raw recovery progress to cat recovery API #17064 (issue: #17022)
• Remove host from cat nodes API #16656 (issues: #12959, #16575)
• Using the accept header in the request instead of content-type in _cat API. #14421 (issue: #14195)

CRUD

• Fixed naming inconsistency for fields/stored_fields in the APIs #20166 (issues: #18943, #20155)
• Disallow creating indices starting with - or + #20033 (issue: #19800)
• Wait for changes to be visible by search #17986 (issue: #1063)
• Remove object notation for core types. #15684 (issue: #15388)

Cache

• Remove deprecated query cache settings #15592

Cluster

• Persistent Node Ids #19140 (issue: #17811)
• Remove validation errors from cluster health response #17773 (issue: #16979)
• Remove memory section #17278 (issues: #12049, #16756)

Core

• Remove ignore system bootstrap checks #20511
• Remove minimum master nodes bootstrap check #20082
• Keep input time unit when parsing TimeValues #19102
• Remove cluster name from data path #18554 (issue: #17810)
• Add max number of processes check #16919
• Add mlockall bootstrap check #16909
• One log #16703 (issue: #16585)

Engine

• Optimize indexing for the autogenerated ID append-only case #20211 (issue: #19813)
• Remove index.compound_on_flush setting and default to true #15594 (issue: #10778)

Exceptions

• Die with dignity #19272 (issue: #19231)

Fielddata

• Remove "uninverted" and "binary" fielddata support for numeric and boolean fields. #14082 (issue: #14113)

Geo

• Deprecate GeoDistance enums and remove geo distance script helpers #19783

Index APIs

• Removes write consistency level across replication action APIs in favor of wait_for_active_shards #19454 (issue: #18985)
• Remove GET option for /_forcemerge #15223 (issue: #15165)
• Remove /_optimize REST API endpoint #14226 (issue: #13778)

Indexed Scripts/Templates

• Store indexed scripts in the cluster state instead of the .scripts index #17650 (issue: #16651)

Inner Hits

• Also do not serialize _index key in search response for parent/child inner hits #19011
• Don’t include _id, _type and _index keys in search response for inner hits #18995 (issue: #18091)
• Nested inner hits shouldn’t use relative paths #18567 (issue: #16653)
• Drop top level inner hits in favour of inner hits defined in the query dsl #17816 (issue: #11118)

Internal

• _flush should block by default #20597 (issue: #20569)
• Actually bound the generic thread pool #17017
• Remove support for pre 2.0 indices #13799

Logging

• Introduce Log4j 2 #20235 (issues: #16030, #17697)

Mapping

• Remove _timestamp and _ttl on 5.x indices. #18980 (issue: #18280)
• Add a soft limit on the mapping depth. #17400
• Disable fielddata on text fields by defaults. #17386
• Add limit to total number of fields in mapping #17357
• Change the field mapping index time boost into a query time boost. #16900
• Deprecate string in favor of text/keyword. #16877
• Term vector APIs should no longer update mappings #16285
• Remove the format option of the _source field. #15398
• Remove transform #13657 (issue: #12674)

Packaging

• Rename service.bat to elasticsearch-service.bat #20496 (issue: #17528)
• Remove -D handling in args for windows plugin script #20378 (issue: #18207)
• Set default min heap equal to default max heap #20080 (issues: #16334, #17728, #18311)
• Remove allow running as root #18694 (issue: #18688)
• Require /bin/bash in packaging #18259 (issue: #18251)
• Remove plugin script parsing of system properties #18207 (issue: #18140)
• Add JVM options configuration file #17675 (issue: #17121)

Parent/Child

• Removed total score mode in favour for sum score mode. #17174 (issues: #13470, #17083)
• Removed pre 2.x parent child implementation #13376

Percolator

• Remove .percolator type in favour of percolator field type #17560
• Change the percolate api to not dynamically add fields to mapping #16077 (issue: #15751)

Plugin Delete By Query

• Remove Delete-By-Query plugin #18516 (issue: #18469)

Plugin Lang Painless

• Remove all date now methods from Painless #20766 (issue: #20762)
• Make Painless the Default Language #20017 (issue: #19960)

Plugins

• Plugins cleanup #18594 (issue: #18588)
• Rename bin/plugin in bin/elasticsearch-plugin #16454
• Change the inner structure of the plugins zip #16453
• Remove multicast plugin #16326 (issue: #16310)
• Plugins: Remove site plugins #16038

Query DSL

• Lessen leniency of the query dsl. #18276
• Function score query: remove deprecated support for boost_factor #13510
• Remove support for deprecated queries. #13418 (issue: #13326)

REST

• Change separator for shards preference #20786 (issues: #20722, #20769)
• Parameter improvements to Cluster Health API wait for shards #20223 (issue: #20216)
• Switch indices.exists_type from {index}/{type} to {index}/_mapping/{type}. #20055 (issue: #15613)
• Only use PUT for index creation, not POST. #20001 (issues: #15613, #18160)
• Remove camelCase support #17933 (issue: #8988)
• Remove case parameter from rest apis #17774 (issue: #8988)
• Disallow unquoted field names #15351 (issue: #9800)
• Limit the accepted length of the _id #16036 (issue: #16034)

Scripting

• Hardcode painless as the default scripting lang and add legacy script default for stored scripts #20310 (issue: #20122)
• Remove deprecated 1.x script and template syntax #19387 (issue: #13729)
• Allow only a single extension for a scripting engine #18332 (issue: #10598)
• Remove sandbox option for script settings, allow only registering a single language. #18226 (issues: #10598, #17114)

Search

• Rename fields to stored_fields and add docvalue_fields #18992 (issue: #18943)
• Remove only node preference #18875 (issue: #18822)
• Add search preference to prefer multiple nodes #18872 (issue: #18822)
• Add a soft limit on the number of shards that can be queried in a single search request. #17396
• Remove deprecated reverse option from sorting #17282 (issue: #17047)
• Remove some deprecations #14331
• Remove search exists api #13911 (issues: #13682, #13910)
• Remove the scan and count search types. #13310

Search Refactoring

• Remove deprecated parameter from field sort builder. #16573 (issue: #16127)
• Remove "query" query and fix related parsing bugs #14304 (issue: #13326)

Settings

• Default max local storage nodes to one #19964 (issues: #19679, #19748)
• Persistent Node Names #19456 (issue: #19140)
• Remove support for properties #19398 (issues: #19388, #19391)
• Rename boostrap.mlockall to bootstrap.memory_lock #18669
• Register indices.query.bool.max_clause_count setting #18341 (issue: #18336)
• Remove settings and system properties entanglement #18198 (issue: #18197)
• Remove action.get.realtime setting #17857 (issue: #12543)
• Remove ability to specify arbitrary node attributes with node. prefix #17402 (issue: #17280)
• Enforce discovery.zen.minimum_master_nodes is set when bound to a public ip #17288
• Prevent index level setting from being configured on a node level #17144 (issue: #16799)
• Remove support for node.client setting #16963 (issue: #16565)
• Remove es.max-open-files flag #16757 (issues: #16506, #483)
• Enforce node level limits if node is started in production env #16733 (issue: #16727)
• Make settings validation strict #16365
• Remove the ability to fsync on every operation and only schedule fsync task if really needed #16257 (issue: #16152)
• Script settings #16197
• Remove index.flush_on_close entirely #15977
• Restore chunksize of 512kb on recovery and remove configurability #15235 (issue: #15161)
• Remove ancient deprecated and alternative recovery settings #15234

Similarities

• Renames default similarity into classic #15446 (issue: #15102)

Snapshot/Restore

• Change the default of include_global_state from true to false for snapshot restores #18773 (issue: #18569)
• Fail closing or deleting indices during a full snapshot #17021 (issue: #16321)

Stats

• Modify load average format #15932 (issue: #15907)
• Reintroduce five-minute and fifteen-minute load averages on Linux #15907 (issues: #12049, #14741)
• Add system CPU percent to OS stats #14741

Term Vectors

• Remove DFS support from TermVector API #16452

Translog

• Drop support for simple translog and hard-wire buffer to 8kb #15574
• Simplify translog-based flush settings #15573

Warmers

• Remove query warmers and the warmer API. #15614 (issue: #15607)

Breaking Java changes

Aggregations

• getKeyAsString and key_as_string should be the same for terms aggregation on boolean field #15393

Allocation

• Move parsing of allocation commands into REST and remove support for plugins to register allocation commands #17802 (issue: #17894)
• Simplify shard balancer interface #17028 (issue: #8954)

Analysis

• Simplify Analysis registration and configuration #14355

CRUD

• Removing isCreated and isFound from the Java API #19645 (issues: #19566, #19631)

Cache

• Refactor IndicesRequestCache to make it testable. #16610
• Fold IndexCacheModule into IndexModule #14293

Core

• Remove ability to plug-in TransportService #20505
• Register thread pool settings #18674 (issues: #18613, #9216)
• Bootstrap does not set system properties #17088 (issues: #16579, #16791)
• Remove es.useLinkedTransferQueue #16786

Discovery

• Introduce node handshake #15971 (issue: #9061)
• Include pings from client nodes in master election #17329 (issue: #17325)

Highlighting

• Register Highlighter instances instead of classes #18859

Internal

• Remove TransportService#registerRequestHandler leniency #20469 (issue: #20468)
• Consolidate search parser registries #20000
• Move all FetchSubPhases to o.e.search.fetch.subphase #19702
• Squash the rest of o.e.rest.action #19698
• Clean up BytesReference #19196
• Cleanup ClusterService dependencies and detached from Guice #18941
• Simplify SubFetchPhase interface #18881
• Simplify FetchSubPhase registration and detach it from Guice #18862
• Remove duplicate getters from DiscoveryNode and DiscoveryNodes #17410 (issue: #16963)
• Cli: Switch to jopt-simple #17024 (issue: #11564)
• Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation #15776
• Remove NodeBuilder #15354
• Fix IndexSearcherWrapper interface to not depend on the EngineConfig #14654
• Cleanup query parsing and remove IndexQueryParserService #14452
• Remove circular dependency between IndicesService and IndicesStore #14285
• Remove guice injection from IndexStore and friends #14279
• Replace IndicesLifecycle with a per-index IndexEventListener #14217 (issue: #13259)
• Simplify similarity module and friends #13942
• Refactor SearchRequest to be parsed on the coordinating node #13859

Java API

• Add a dedicated client/transport project for transport-client #19435 (issue: #19412)
• Remove setRefresh #18752 (issue: #1063)
• Remove the count api #14166 (issue: #13928)
• IdsQueryBuilder to accept only non null ids and types #13937

Mapping

• [Mapping] Several MappingService cleanups #16133 (issue: #15924)

Network

• Factor out abstract TCPTransport* classes to reduce the netty footprint #19096
• Remove ability to disable Netty gathering writes #16774 (issue: #7811)

Parent/Child

• Cleanup ParentFieldMapper #16045
• Several other parent/child cleanups #13470

Percolator

• Move the percolator from core to its own module #18511
• Remove percolator cache #18434

Plugins

• Cleanup sub fetch phase extension point #20382
• Remove IndexTemplateFilter #20072
• Switch custom ShardsAllocators to pull based model #20071
• Make custom allocation deciders use pull based extensions #20040
• Migrate query registration from push to pull #19376
• Add components getter as bridge between guice and new plugin init world #19371
• Remove CustomNodeAttributes extension point #19348
• Add RepositoryPlugin interface for registering snapshot repositories #19324
• Simplified repository api for snapshot/restore #19292
• Switch most search extensions from push to pull #19238
• Move RestHandler registration to ActionModule and ActionPlugin #19165
• Pull actions from plugins #19108
• Switch analysis from push to pull #19073
• Remove guice from Mapper plugins #19018
• Fail to start if plugin tries broken onModule #19025
• Simplify ScriptModule and script registration #18903
• Cut over settings registration to a pull model #18890
• Enforce isolated mode for all plugins #17276
• Don’t use guice for QueryParsers #15761
• Remove guice from the index level #14518
• Remove shard-level injector #13881

Query DSL

• Remove the MissingQueryBuilder which was deprecated in 2.2.0. #15364 (issue: #14112)
• Remove NotQueryBuilder #14204 (issue: #13761)

Scripting

• Remove o.e.script.Template class and move template query to lang-mustache module #19425 (issue: #16314)
• Move search template to lang-mustache module #18765 (issue: #17906)
• Remove LeafSearchScript.runAsFloat(): Nothing calls it. #18364

Search

• Remove FetchSubPhaseParseElement #20350
• Refactor of query profile classes to make way for other profile implementations #18370 (issue: #10538)
• Query refactoring: split parse phase into fromXContent and toQuery for all queries #13788 (issue: #10217)

Search Refactoring

• Refactored inner hits parsing and intoduced InnerHitBuilder #17291
• Remove support for query_binary and filter_binary #14433 (issue: #14308)
• Validate query api: move query parsing to the coordinating node #14384

Settings

• Remove node.mode and node.local settings #19428
• Remove Settings.settingsBuilder. #17619
• Move remaining settings in NettyHttpServerTransport to the new infra #16531
• Replace IndexSettings annotation with a full-fledged class #14251
• Fix ping timeout settings inconsistencies #13701 (issue: #6579)

Snapshot/Restore

• Removes extra writeBlob method in BlobContainer #19727 (issue: #18528)

Store

• Standardize state format type for global and index level metadata #17123

Suggesters

• Remove suggest threadpool #17304 (issue: #17198)
• Remove suggest transport action #17198 (issue: #10217)

Deprecations

CRUD

• Deprecate found and created in delete and index rest responses #19633

Plugin Discovery Azure Classic

• Deprecate discovery-azure and rename it to discovery-azure-classic #19186 (issue: #19144)

Plugin Mapper Attachment

• Deprecate mapper-attachments plugin #16948 (issue: #16910)

Query DSL

• Deprecate Indices query #17710 (issue: #12017)
• Deprecate mlt, in and geo_bbox query name shortcuts #17507

Query Refactoring

• Splits phrase and phrase_prefix in match query into MatchPhraseQueryBuilder and MatchPhrasePrefixQueryBuilder #17508

Scripting

• Deprecate Groovy, Python, and Javascript #20244 (issue: #20129)

Search

• Deprecate fuzzy query #16211 (issues: #15760, #16121)

Templates

• Deprecate template query #19607 (issue: #19390)

New features

Aggregations

• Split regular histograms from date histograms. #19551 (issues: #4847, #8082)
• Adds aggregation profiling to the profile API #18414 (issue: #10538)
• New Matrix Stats Aggregation module #18300 (issue: #16826)

Aliases

• Add an alias action to delete an index #20184 (issue: #20064)

Allocation

• Add API to explain why a shard is or isn’t assigned #17305 (issue: #14593)

Analysis

• Exposing lucene 6.x minhash filter. #20206 (issue: #20149)
• Add fingerprint token filter and fingerprint analyzer #17873 (issue: #13325)

Circuit Breakers

• Circuit break on aggregation bucket numbers with request breaker #19394 (issue: #14046)

Discovery

• Add two phased commit to Cluster State publishing #13062

Geo

• Cut over geo_point field and queries to new LatLonPoint type #20315 (issue: #20314)

Index APIs

• Add rollover API to switch index aliases given some predicates #18732 (issue: #18647)

Ingest

• ingest-useragent plugin #19074
• Add a Sort ingest processor #17999
• Add date_index_name processor #17973 (issue: #17814)
• Merge feature/ingest branch into master branch #16049 (issue: #14049)

Java REST Client

• Introduce async performRequest method #19400
• Low level Rest Client #18735 (issue: #7743)

Mapping

• Add scaled_float. #19264 (issues: #15939, #1941)
• Expose half-floats. #18887
• Add a text field. #16637
• Add a new keyword field. #16589 (issue: #14113)

Percolator

• index the query terms from the percolator query #13646 (issue: #12664)

Plugin Analysis ICU

• Adding support for customizing the rule file in ICU tokenizer #13651 (issue: #13146)

Plugin Discovery File

• File-based discovery plugin #20394 (issue: #20323)

Plugin Ingest Attachment

• Ingest: Add attachment processor #16490 (issue: #16303)

Plugin Mapper Attachment

• Migrate mapper attachments plugin to main repository #14605

Plugin Repository HDFS

• HDFS Snapshot/Restore plugin #15192 (issue: #15191)

Plugin Repository S3

• Add support for path_style_access #15114

Query DSL

• Adds a rewrite phase to queries on the shard level #16870 (issue: #9526)

Reindex API

• Reindex from remote #18585 (issue: #17447)
• Port Delete By Query to Reindex infrastructure #18329 (issue: #16883)
• Merge reindex to master #16861

Scripting

• Exceptions and Infinite Loop Checking #15936
• Added a new scripting language (PlanA) #15136 (issue: #13084)

Scroll

• Add the ability to partition a scroll in multiple slices. #18237 (issue: #13494)

Search

• Add the ability to disable the retrieval of the stored fields entirely #20026
• Add search_after parameter in the SearchAPI #16125 (issue: #8192)

Settings

• Add infrastructure to transactionally apply and reset dynamic settings #15278

Snapshot/Restore

• Add Google Cloud Storage repository plugin #13578 (issue: #12880)

Stats

• Extend field stats to report searchable/aggregatable fields #17980 (issue: #17750)
• API for listing index file sizes #16661 (issue: #16131)

Store

• Expose MMapDirectory.preLoad(). #18880
• Add primitive to shrink an index into a single shard #18270

Suggesters

• Add support for returning documents with completion suggester #19536 (issue: #10746)
• Add document-oriented completion suggester #14410 (issue: #10746)

Task Manager

• Add task cancellation mechanism #16320
• Make the Task object available to the action caller #16033
• Task Management: Add framework for registering and communicating with tasks #15347 (issue: #15117)

Translog

• Add elasticsearch-translog CLI tool with truncate command #19342 (issue: #19123)

Enhancements

Aggregations

• Make the heuristic to compute the default shard size less aggressive. #19659
• Add _bucket_count option to buckets_path #19571 (issue: #19553)
• Remove AggregationStreams #19507
• Migrate serial_diff aggregation to NamedWriteable #19483
• Migrate most remaining pipeline aggregations to NamedWriteable #19480
• Migrate moving_avg pipeline aggregation to NamedWriteable #19420
• Migrate matrix_stats to NamedWriteable #19418
• Migrate derivative pipeline aggregation to NamedWriteable #19407
• Migrate top_hits, histogram, and ip_range aggregations to NamedWriteable #19375
• Migrate nested, reverse_nested, and children aggregations to NamedWriteable #19374
• Migrate geohash_grid and geo_bounds aggregations to NamedWriteable #19372
• Clean up significant terms aggregation results #19365
• Migrate range, date_range, and geo_distance aggregations to NamedWriteable #19321
• Migrate terms aggregation to NamedWriteable #19277
• Migrate sampler and missing aggregations to NamedWriteable #19259
• Migrate global, filter, and filters aggregation to NamedWriteable #19220
• Migrate the cardinality, scripted_metric, and geo_centroid aggregations to NamedWriteable #19219
• Use a static default precision for the cardinality aggregation. #19215
• Migrate more aggregations to NamedWriteable #19199
• Migrate stats and extended stats to NamedWriteable #19198
• Migrate sum, min, and max aggregations over to NamedWriteable #19194
• Start migration away from aggregation streams #19097
• Automatically set the collection mode to breadth_first in the terms aggregation when the cardinality of the field is unknown or smaller than the requested size. #18779 (issue: #9825)
• Rename PipelineAggregatorBuilder to PipelineAggregationBuilder. #18677 (issue: #18377)
• AggregatorBuilder and PipelineAggregatorBuilder do not need generics. #18368 (issue: #18133)
• Rename AggregatorBuilder to AggregationBuilder #18377 (issue: #18367)
• Add the ability to use the breadth_first mode with nested aggregations (such as top_hits) which require access to score information. #18127 (issue: #9825)
• Make significant terms work on fields that are indexed with points. #18031
• Add tests and documentation for using time_zone in date range aggregation #16955 (issue: #10130)
• Fixes serialisation of Ranges #16674

Allocation

• Verify AllocationIDs in replication actions #20320
• Mark shard as stale on non-replicated write, not on node shutdown #20023 (issue: #18919)
• Add routing changes API to RoutingAllocation #19992
• Primary shard allocator observes limits in forcing allocation #19811 (issue: #19446)
• Use primary terms as authority to fail shards #19715
• Add recovery source to ShardRouting #19516
• Allow _shrink to N shards if source shards is a multiple of N #18699
• Only filter intial recovery (post API) when shrinking an index #18661
• Estimate shard size for shrinked indices #18659
• Only fail relocation target shard if failing source shard is a primary #18574 (issue: #16144)
• Simplify delayed shard allocation #18351 (issue: #18293)
• Limit retries of failed allocations per index #18467 (issue: #18417)
• Immutable ShardRouting #17821
• Add the shard’s store status to the explain API #17689 (issue: #17372)
• Write shard state metadata as soon as shard is created / initializing #16625 (issue: #14739)
• Reuse existing allocation id for primary shard allocation #16530 (issue: #14739)
• Remove version in ShardRouting (now obsolete) #16243 (issue: #14739)
• Prefer nodes that previously held primary shard for primary shard allocation #16096 (issue: #14739)
• Extend reroute with an option to force assign stale primary shard copies #15708 (issue: #14739)
• Allocate primary shards based on allocation IDs #15281 (issue: #14739)
• Persist currently started allocation IDs to index metadata #14964 (issue: #14739)
• Use ObjectParser to parse AllocationID #14962 (issue: #14831)
• Persist allocation ID with shard state metadata on nodes #14831 (issue: #14739)

Analysis

• Stop using cached component in _analyze API #19929 (issue: #19827)
• Specify custom char_filters/tokenizer/token_filters in the analyze API #15671 (issue: #8878)
• Add a MultiTermAwareComponent marker interface to analysis factories. #19028 (issues: #18064, #9978)
• Add Flags Parameter for Char Filter #18363 (issue: #18362)
• Core: better error message when analyzer created without tokenizer or… #18455 (issue: #15492)
• Move AsciiFolding earlier in FingerprintAnalyzer filter chain #18281 (issue: #18266)
• Improve error message if resource files have illegal encoding #17237 (issue: #17212)

Benchmark

• Add client-benchmark-noop-api-plugin to stress clients even more in benchmarks #20103

CAT API

• Add health status parameter to cat indices API #20393
• Includes the index UUID in the _cat/indices API #19204 (issue: #19132)
• Add node name to Cat Recovery #18187 (issue: #8041)
• Add support for documented byte/size units and for micros as a time unit in _cat API #17779
• Add _cat/tasks #17551
• Cat health supports ts=0 option #13508 (issue: #10109)
• Expose http address in cat/nodes #16770
• [cat/recovery] Make recovery time a TimeValue() #16743 (issue: #9209)
• :CAT API: remove space at the end of a line #15250 (issue: #9464)

CRUD

• Renaming operation to result and reworking responses #19704 (issue: #19664)
• Adding _operation field to index, update, delete response. #19566 (issues: #19267, #9642, #9736)
• CRUD: Allow to get and set ttl as a time value/string #15047

Cache

• Enable option to use request cache for size > 0 #19472
• Cache FieldStats in the request cache #18768 (issue: #18717)
• Allow the query cache to be disabled. #16268 (issue: #15802)
• Enable the indices request cache by default #17162 (issues: #16870, #17134)

Circuit Breakers

• Cluster Settings Updates should not trigger circuit breakers. #20827
• Circuit break the number of inline scripts compiled per minute #19694 (issue: #19396)

Cluster

• Add clusterUUID to RestMainAction output #20503
• Batch process node left and node failure #19289 (issue: #19282)
• Index creation waits for write consistency shards #18985
• Inline reroute with process of node join/master election #18938 (issue: #17270)
• Index creation does not cause the cluster health to go RED #18737 (issues: #9106, #9126)
• Cluster Health class improvements #18673
• Adds tombstones to cluster state for index deletions #17265 (issues: #16358, #17435)
• Enable acked indexing #17038 (issue: #7572)
• Cluster Health should run on applied states, even if waitFor=0 #17440
• Resolve index names to Index instances early #17048
• Remove DiscoveryNode#shouldConnectTo method #16898 (issue: #16815)
• Fail demoted primary shards and retry request #16415 (issue: #14252)
• Illegal shard failure requests #16275
• Shard failure requests for non-existent shards #16089 (issue: #14252)
• Add handling of channel failures when starting a shard #16041 (issue: #15895)
• Wait for new master when failing shard #15748 (issue: #14252)
• Master should wait on cluster state publication when failing a shard #15468 (issue: #14252)
• Split cluster state update tasks into roles #14899 (issue: #13627)
• Add timeout mechanism for sending shard failures #14707 (issue: #14252)
• Add listener mechanism for failures to send shard failed #14295 (issue: #14252)

Core

• Add production warning for pre-release builds #20674
• Add serial collector bootstrap check #20558
• Rename Netty TCP transports thread factories from http_* to transport_* #20207
• Do not log full bootstrap checks exception #19989
• Mark halting the virtual machine as privileged #19923 (issues: #19272, #19806)
• Makes index creation more friendly #19450 (issue: #9126)
• Clearer error when handling fractional time values #19158 (issue: #19102)
• Read Elasticsearch manifest via URL #18999 (issue: #18996)
• Throw if the local node is not set #18963 (issue: #18962)
• Bootstrap check for OnOutOfMemoryError and seccomp #18756 (issue: #18736)
• Log OS and JVM on startup #18557
• Add GC overhead logging #18419
• Refactor JvmGcMonitorService for testing #18378
• Default to server VM and add client VM check #18155
• Add system bootstrap checks escape hatch #18088
• Avoid sliced locked contention in internal engine #18060 (issue: #18053)
• Add heap size bootstrap check #17728 (issue: #17490)
• Remove hostname from NetworkAddress.format #17601 (issue: #17604)
• Bootstrapping bootstrap checks #17595 (issues: #17474, #17570)
• Add max map count check #16944
• Remove PROTOTYPE from BulkItemResponse.Failure #17433 (issue: #17086)
• Throw an exception if Writeable.Reader reads null #17332
• Remove PROTOTYPE from RescorerBuilders #17330
• Port Primary Terms to master #17044 (issues: #14062, #14651, #17038)
• Use index UUID to lookup indices on IndicesService #17001
• Add -XX+AlwaysPreTouch JVM flag #16937
• Add max size virtual memory check #16935
• Use and test relative time in TransportBulkAction #16916
• Bump Elasticsearch version to 5.0.0-SNAPSHOT #16862
• Assert that we can write in all data-path on startup #16745
• Add G1GC check on startup #16737 (issue: #10740)
• Shards with heavy indexing should get more of the indexing buffer #14121
• Remove and ban ImmutableMap #13939 (issue: #13224)
• Finish banning ImmutableSet #13820 (issue: #13224)
• Removes and bans ImmutableSet #13754 (issue: #13224)
• Remove and ban ImmutableMap#entrySet #13724
• Forbid ForwardingSet #13720 (issue: #13224)

Dates

• Improve TimeZoneRoundingTests error messages #18895
• Support full range of Java Long for epoch DateTime #18509 (issue: #17936)

Discovery

• Do not log cluster service errors at after joining a master #19705
• Log warning if minimum_master_nodes set to less than quorum #15625
• Add a dedicate queue for incoming ClusterStates #13303 (issue: #13062)

Engine

• Only try to read new segments info if we really flushed the index #20474
• Use _refresh instead of reading from Translog in the RT GET case #20102
• Remove writeLockTimeout from InternalEngine #16930
• Don’t guard IndexShard#refresh calls by a check to isRefreshNeeded #16118
• Never call a listerner under lock in InternalEngine #15786
• Use System.nanoTime() to initialize Engine.lastWriteNanos #14321
• Flush big merges automatically if shard is inactive #14275
• Remove Engine.Create #13955
• Remove the disabled autogenerated id optimization from InternalEngine #13857

Exceptions

• Improve startup exception #20083
• Make NotMasterException a first class citizen #19385
• Do not catch throwable #19231
• Make the index-too-old exception more explicit #18438
• Add index name in IndexAlreadyExistsException default message #18274
• Fix typos in exception/assert/log messages in core module. #16649
• Add field names to several mapping errors #16508 (issue: #16378)
• Add serialization support for more important IOExceptions #15766
• Adds exception objects to log messages. #14827 (issue: #10021)
• Add stack traces to logged exceptions where missing #13825 (issue: #10021)
• Remove reflection hacks from ElasticsearchException #13796
• Rename QueryParsingException to a more generic ParsingException #13631
• Add *Exception(Throwable cause) constructors/ call where appropriate #13544 (issue: #10021)

Expressions

• improve date api for expressions/painless fields #18658
• Support geo_point fields in lucene expressions #18096
• Add support for .empty to expressions, and some docs improvements #18077

Geo

• GeoBoundingBoxQueryBuilder should throw IAE when topLeft and bottomRight are the same coordinate #18668 (issues: #18458, #18631)
• Enhanced lat/long error handling #16833 (issue: #16137)
• Fix a potential parsing problem in GeoDistanceSortParser #17111
• Geo: Add validation of shapes to ShapeBuilders #15551 (issue: #14416)
• Make remaining ShapeBuilders implement Writeable #15010 (issue: #14416)
• Geo: Remove internal translated flag from LineStringBuilder #14969
• Make PointBuilder, CircleBuilder & EnvelopeBuilder implement Writable #14933 (issue: #14416)
• Merging BaseLineString and BasePolygonBuilder with subclass #14887 (issue: #14482)
• Moving static factory methods to ShapeBuilders #14529
• Remove InternalLineStringBuilder and InternalPolygonBuilder #14482 (issue: #14416)

Highlighting

• Switch Highlighting to ObjectParser #17363
• Use HighlightBuilder in SearchSourceBuilder #15376 (issue: #15044)
• Joint parsing of common global Hightlighter and subfield parameters #15368 (issue: #15285)
• Enable HighlightBuilder to create SearchContextHighlight #15324
• Add fromXContent method to HighlightBuilder #15157

Index APIs

• Add date-math support to _rollover #20709
• Add Shrink request source parser to parse create index request body #18802
• Fail hot_threads in a better way if unsupported by JDK #15909

Index Templates

• Add "version" field to Templates #20353 (issue: #20171)
• Parse and validate mappings on index template creation #8802 (issue: #2415)

Ingest

• Add "version" field to Pipelines #20343 (issue: #20171)
• Make it possible for Ingest Processors to access AnalysisRegistry #20233
• add ignore_missing option to convert,trim,lowercase,uppercase,grok,rename #20194 (issue: #19995)
• Add support for parameters to the script ingest processor #20136
• introduce the JSON Processor #20128 (issue: #20052)
• Allow rename processor to turn leaf fields into branch fields #20053 (issue: #19892)
• remove ability to set field value in script-processor configuration #19981
• Add REST _ingest/pipeline to get all pipelines #19603 (issue: #19585)
• Show ignored errors in verbose simulate result #19404 (issue: #19319)
• update foreach processor to only support one applied processor. #19402 (issue: #19345)
• Skip the execution of an empty pipeline #19200 (issue: #19192)
• Add ignore_failure option to all ingest processors #18650 (issue: #18493)
• new ScriptProcessor for Ingest #18193
• Expose underlying processor to blame for thrown exception within CompoundProcessor #18342 (issue: #17823)
• Avoid string concatentation in IngestDocument.FieldPath #18108
• add ability to specify multiple grok patterns #18074 (issue: #17903)
• add ability to disable ability to override values of existing fields in set processor #17902 (issue: #17659)
• Streamline option naming for several processors #17892 (issue: #17835)
• add automatic type conversion support to ConvertProcessor #17263 (issue: #17139)
• Give the foreach processor access to the rest of the document #17172 (issue: #17147)
• Added ingest statistics to node stats API #16915
• Add ingest_took to bulk response #16876
• Add ingest info to node info API, which contains a list of available processors #16865
• Use diffs for ingest metadata in cluster state #16847
• hide null-valued metadata fields from WriteableIngestDocument#toXContent #16557
• Ingest: use bulk thread pool for bulk request processing (was index before) #16539 (issue: #16503)
• Add foreach processor #16432
• revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation #16355
• Add processor tags to on_failure metadata in ingest pipeline #16324 (issue: #16202)
• catch processor/pipeline factory exceptions and return structured error responses #16276 (issue: #16010)
• Ingest: move get/put/delete pipeline methods to ClusterAdminClient #16242
• Geoip processor: remove redundant latitude and longitude fields and make location an object with lat and lon subfields #16173

Inner Hits

• Change scriptFields member in InnerHitBuilder to set #18092 (issue: #5831)

Internal

• Remove poor-mans compression in InternalSearchHit and friends #20472
• Don’t register SearchTransportService handlers more than once #20468
• Unguice SearchModule #20456
• Deguice SearchService and friends #20423
• NodeStats classes to implement Writeable rather then Streamable #20327
• More info classes to implement Writeable rather than Streamable #20288 (issue: #20255)
• Internal: Split disk threshold monitoring from decider #20018
• Switching LockObtainFailedException over to ShardLockObtainFailedException #19991 (issue: #19978)
• update and delete by query requests to implement IndicesRequest.Replaceable #19961
• VersionFetchSubPhase should not use Versions#loadDocIdAndVersion #19944
• Remove useless PK lookup in IndicesTTLService #19945
• ignore some docker craziness in seccomp environment checks #19754
• Make Priority an enum #19448
• Snapshot UUIDs in blob names #19421 (issues: #18156, #18815, #19002, #7540)
• Add RestController method for deprecating in one step #19343
• Tighten ensure atomic move cleanup #19309 (issue: #19036)
• Enable checkstyle ModifierOrder #19214
• Expose task information from NodeClient #19189
• Changed rest handler interface to take NodeClient #19170
• Deprecate ExceptionsHelper.detailedMessage #19160 (issue: #19069)
• Factor out ChannelBuffer from BytesReference #19129
• Cleanup Compressor interface #19125
• Hot methods redux #19016 (issue: #16725)
• Remove forked joda time BaseDateTime class #18953
• Support optional ctor args in ConstructingObjectParser #18725
• Remove thread pool from page cache recycler #18664 (issue: #18613)
• Do not automatically close XContent objects/arrays #18549 (issue: #18433)
• Remove use of a Fields class in snapshot responses #18497
• Removes multiple toXContent entry points for SnapshotInfo #18494
• Removes unused methods in the o/e/common/Strings class #18346
• Determine content length eagerly in HttpServer #18203
• Consolidate query generation in QueryShardContext #18129
• Make reset in QueryShardContext private #18113
• Remove Strings#splitStringToArray #18110
• Add toString() to GetResponse #18102
• ConstructingObjectParser adapts ObjectParser for ctor args #17596 (issue: #17352)
• Makes Script type writeable #17908 (issue: #17753)
• FiltersAggregatorBuilder: Don’t create new context for inner parsing #17851
• Clean up serialization on some stats #17832 (issue: #17085)
• Normalize registration for SignificanceHeuristics #17830 (issue: #17085)
• Make (read|write)NamedWriteable public #17829 (issue: #17682)
• Use try-with-resource when creating new parser instances where possible #17822
• Don’t pass XContentParser to ParseFieldRegistry#lookup #17794
• Internal: Remove threadlocal from document parser #17764
• Cut range aggregations to registerAggregation #17757 (issue: #17085)
• Remove ParseFieldMatcher from AbstractXContentParser #17756 (issue: #17417)
• Remove parser argument from methods where we already pass in a parse context #17738
• Switch SearchAfterBuilder to writeGenericValue #17735 (issue: #17085)
• Remove StreamableReader #17729 (issue: #17085)
• Cleanup nested, has_child & has_parent query builders for inner hits construction #17719 (issue: #11118)
• Make AllocationCommands NamedWriteables #17661
• Isolate StreamableReader #17656 (issue: #17085)
• Create registration methods for aggregations similar to those for queries #17653 (issues: #17085, #17389)
• Remove PROTOTYPEs from QueryBuilders #17632 (issue: #17085)
• Remove registerQueryParser #17608
• ParseField#getAllNamesIncludedDeprecated to not return duplicate names #17504
• Rework a query parser and improve registration #17458
• Clean up QueryParseContext and don’t hold it inside QueryRewrite/ShardContext #17417
• Remove PROTOTYPE from MLT.Item #17481 (issue: #17085)
• Remove PROTOTYPE from VersionType #17480 (issue: #17085)
• Remove PROTOTYPEs from highlighting #17466 (issue: #17085)
• Remove PROTOTYPEs from ingest #17434 (issue: #17085)
• Start to rework query registration #17424
• Factor out slow logs into Search and IndexingOperationListeners #17398
• Remove PROTOTYPE from Suggesters #17370
• Remove PROTOTYPE from SortBuilders #17337 (issue: #17085)
• Remove PROTOTYPE from ShapeBuilders #17336 (issue: #17085)
• Replace FieldStatsProvider with a method on MappedFieldType. #17334
• Stop using PROTOTYPE in NamedWriteableRegistry #17284 (issue: #17085)
• Support scheduled commands in current context #17077
• Thread limits #17003
• Remove leniency from segments info integrity checks #16985 (issue: #16973)
• Rename SearchServiceTransportAction to SearchTransportService #16880
• Decouple the TransportService and ClusterService #16872 (issue: #16788)
• Refactor bootstrap checks #16844 (issues: #16733, #16835)
• Add LifecycleRunnable #16752
• Hot inlined methods in your area #16725
• Move IndicesQueryCache and IndicesRequestCache into IndicesService #16603
• Forbid use of java.security.MessageDigest#clone() #16543 (issue: #16479)
• Make IndicesWarmer a private class of IndexService #16470
• Simplify IndicesFieldDataCache and detach from guice #16469
• Uppercase ells (L) in long literals #16329 (issue: #16279)
• ShardId equality and hash code inconsistency #16319 (issue: #16217)
• Ensure all resources are closed on Node#close() #16316 (issue: #13685)
• Make index uuid available in Index, ShardRouting & ShardId #16217
• Move RefreshTask into IndexService and use since task per index #15933
• Make IndexingMemoryController private to IndicesService #15877
• Cleanup IndexingOperationListeners infrastructure #15875
• Remove and forbid use of j.u.c.ThreadLocalRandom #15862 (issue: #15294)
• Fix IntelliJ query builder type inference issues #15429
• Remove and forbid use of Collections#shuffle(List) and Random#<init>() #15299 (issue: #15287)
• Remove and forbid use of the type-unsafe empty Collections fields #15187
• Move IndicesService.canDeleteShardContent to use IndexSettings #15150 (issue: #15059)
• Simplify MonitorService construction and detach from guice #15035
• Use Supplier for StreamInput#readOptionalStreamable #14806
• Add variable-length long encoding #14780
• Extend usage of IndexSetting class #14731 (issue: #14251)
• Fold SimilarityModule into IndexModule #14284
• Move to lucene BoostQuery #14264
• Use built-in method for computing hash code of longs #14213
• Refactor ShardFailure listener infrastructure #14206
• Add methods for variable-length encoding integral arrays #14087
• Fold IndexAliasesService into IndexService #14044
• Remove unneeded Module abstractions #13944
• Query refactoring: simplify IndexQueryParserService parse methods #13938 (issue: #13859)
• Remove and forbid use of com.google.common.collect.Iterators #13916 (issue: #13224)
• Remove and forbid use of com.google.common.collect.ImmutableCollection #13909 (issue: #13224)
• Remove and forbid use of com.google.common.io.Resources #13908 (issue: #13224)
• Remove and forbid use of com.google.common.hash.* #13907 (issue: #13224)
• Remove and forbid use of com.google.common.net.InetAddresses #13905 (issue: #13224)
• Remove and forbid use of com.google.common.collect.EvictingQueue #13903 (issue: #13224)
• Replace Guava cache with simple concurrent LRU cache #13879
• Remove ClusterSerivce and IndexSettingsService dependency from IndexShard #13853
• Start making RecoverySourceHandler unittestable #13840
• Remove IndexService dep. from IndexShard #13797
• Remove ES internal deletion policies in favour of Lucenes implementations #13794
• Move ShardTermVectorService to be on indices level as TermVectorService #13786
• Move ShardPercolateService creation into IndexShard #13777
• Remove ExpressionScriptCompilationException and ExpressionScriptExecutionException #13742
• Reduced the number of ClusterStateUpdateTask variants #13735
• Add a BaseParser helper for stream parsing #13615
• Remove and forbid use of com.google.common.primitives.Ints #13596 (issue: #13224)
• Remove and forbid use of com.google.common.math.LongMath #13575 (issue: #13224)
• Remove and forbid use of com.google.common.base.Joiner #13572 (issue: #13224)
• Replace and ban next batch of Guava classes #13562 (issue: #13224)
• Remove and forbid use of com.google.common.collect.Iterables #13559 (issue: #13224)
• Replace LoadingCache usage with a simple ConcurrentHashMap #13552 (issue: #13224)
• Use Supplier instead of Reflection #13545
• Remove and forbid use of com.google.common.base.Preconditions #13540 (issue: #13224)
• Remove and forbid use of guava Function, Charsets, Collections2 #13533 (issue: #13224)
• Remove and forbid use of com.google.common.collect.ImmutableSortedMap #13525 (issue: #13224)
• Remove and forbid use of several com.google.common.util. classes #13524 (issue: #13224)
• Cleanup SearchRequest & SearchRequestBuilder #13518
• Remove and forbid use of com.google.common.collect.Queues #13498 (issue: #13224)
• Remove and forbid use of com.google.common.base.Preconditions#checkNotNull #13493 (issue: #13224)
• Remove and forbid use of com.google.common.collect.Sets #13463 (issue: #13224)
• Remove and forbid use of com.google.common.collect.Maps #13438 (issue: #13224)
• Remove use of underscore as an identifier #13353
• Remove and forbid the use of com.google.common.base.Predicate(s)? #13349 (issues: #13224, #13314)
• This commit removes com.google.common.io #13302 (issue: #13224)

Java API

• Ensure PutMappingRequest.buildFromSimplifiedDef input are pairs #19837 (issue: #19836)
• Start from a random node number so that clients do not overload the first node configured #14143
• Switch QueryBuilders to new MatchPhraseQueryBuilder #18753
• Improve adding clauses to span_near and span_or query #18485 (issue: #18478)
• QueryBuilder does not need generics. #18133
• Remove copy constructors from request classes and TransportMessage type #16640 (issue: #15776)

Java REST Client

• Add support for a RestClient path prefix #20190
• Add "Async" to the end of each Async RestClient method #20172 (issue: #20168)
• Allow RestClient to send array-based headers #20151
• Add response body to ResponseException error message #19653 (issue: #19653)
• Simplify Sniffer initialization and automatically create the default HostsSniffer #19599
• Remove duplicate dependency declaration for http client #19580 (issue: #19281)
• Add callback to customize http client settings #19373
• Rest Client: add short performRequest method variants without params and/or body #19340 (issue: #19312)

Logging

• Ensure logging is initialized in CLI tools #20575
• Give useful error message if log config is missing #20493
• Complete Elasticsearch logger names #20457 (issue: #20326)
• Add node name to decider trace logging #20437 (issue: #20379)
• Logging shutdown hack #20389 (issue: #20304)
• Disable console logging #20387
• Skip loading of jansi from log4j2 #20334
• Configure AWS SDK logging configuration #20313 (issue: #20294)
• Warn if unsupported logging configuration present #20309
• Size limit deprecation logs #20287 (issues: #20235, #20254)
• Increase visibility of deprecation logger #20254 (issue: #11033)
• Add log message about enforcing bootstrap checks #19451
• Improve logging for batched cluster state updates #19255
• Send HTTP Warning Header(s) for any Deprecation Usage from a REST request #17804 (issue: #17687)
• Throw IllegalStateException when handshake fails due to version or cluster mismatch #18676

Mapping

• Automatically downgrade text and keyword to string on indexes imported from 2.x #20177
• Do not parse numbers as both strings and numbers when not included in _all. #20167
• Don’t index the _version field #20132
• The root object mapper should support updating numeric_detection, date_detection and dynamic_date_formats. #20119 (issue: #20111)
• Automatically upgrade analyzed string fields that have index_options or position_increment_gap set. #20002 (issue: #19974)
• Mappings: Support dots in field names in mapping parsing #19899 (issue: #19443)
• Save one utf8 conversion in KeywordFieldMapper. #19867
• Do not parse the created version from the settings every time a field is parsed. #19824
• Elasticsearch should reject dynamic templates with unknown match_mapping_type. #17285 (issue: #16945)
• Upgrade string fields to text/keyword even if include_in_all is set. #19004 (issue: #18974)
• Adds a methods to find (and dynamically create) the mappers for the parents of a field with dots in the field name #18106 (issue: #15951)
• Automatically upgrade analyzed strings with an analyzer to text. #17861
• Support dots in field names when mapping already exists #17759 (issue: #15951)
• Use the new points API to index numeric fields. #17746 (issues: #11513, #16751, #17007, #17700)
• Simplify AllEntries, AllField and AllFieldMapper: #17613
• Make parseMultiField part of parseField. #17313
• Automatically add a sub keyword field to string dynamic mappings. #17188
• Remove friction from the mapping changes in 5.0. #16991
• Rework norms parameters for 5.0. #16987
• Moved dynamic field handling in doc parsing to end of parsing #16798
• Remove the MapperBuilders utility class. #16609
• Make the index property a boolean. #16161
• Remove the ability to enable doc values with the fielddata.format setting. #16147
• Be stricter about parsing boolean values in mappings. #16146
• Fix default doc values to be enabled when a field is not indexed. #16141
• Dynamically map floating-point numbers as floats instead of doubles. #15319 (issue: #13851)
• Simplify MetaDataMappingService. #15217
• Remove MergeMappingException. #15177

Network

• Avoid early initializing Netty #19819 (issue: #5644)
• Network: Allow to listen on virtual interfaces. #19568 (issues: #17473, #19537)
• Explicitly tell Netty to not use unsafe #19786 (issues: #19562, #5624)
• Enable Netty 4 extensions #19767 (issue: #19526)
• Modularize netty #19392
• Simplify TcpTransport interface by reducing send code to a single send method #19223
• Do not start scheduled pings until transport start #18702

Packaging

• Add quiet option to disable console logging #20422 (issues: #15315, #16159, #17220)
• Explicitly disable Netty key set replacement #20249
• Remove explicit parallel new GC flag #18767
• Use JAVA_HOME or java.exe in PATH like the Linux scripts do #18685 (issue: #4913)
• Don’t mkdir directly in deb init script #18503 (issue: #18307)
• Increase default heap size to 2g #18311 (issues: #16334, #17686, #18309)
• Switch init.d scripts to use bash #18308 (issue: #18259)
• Switch scripts to use bash #18251 (issue: #14002)
• Further simplifications of plugin script #18239 (issue: #18207)
• Pass ES_JAVA_OPTS to JVM for plugins script #18140 (issue: #16790)
• Remove unnecessary sleep from init script restart #17966
• Explicitly set packaging permissions #17912 (issue: #17634)
• rpm uses non-portable --system flag to useradd #14596 (issue: #14211)
• Adding JAVA_HOME to documents and env config file #11338 (issue: #11291)
• Added RPM metadata #17477
• Elasticsearch ownership for data, logs, and configs #17197 (issue: #12688)
• Fail early on JDK with compiler bug #16418 (issues: #16097, #16362)
• Make security non-optional #16176
• Remove RuntimePermission("accessDeclaredMembers") #15378
• Remove Guava as a dependency #14055 (issue: #13224)
• Remove Guava as a dependency #14054 (issue: #13224)

Percolator

• Also support query term extract for queries wrapped inside a FunctionScoreQuery #19184
• Add support for synonym query to percolator query term extraction #19066
• Add percolator query extraction support for dismax query #18845
• Improve percolate query performance by not verifying certain candidate matches #18696
• Improve percolator query term extraction #18610
• PercolatorQueryBuilder cleanup by using MemoryIndex#fromDocument(…) helper #17669 (issue: #9386)
• Add scoring support to the percolator query #17385 (issue: #13827)
• Add query extract support for the blended term query and the common terms query #17347
• Add support for several span queries in ExtractQueryTermsService #17323
• Add support for TermsQuery in ExtractQueryTermsService #17316
• Replace percolate APIs with a percolator query #16349 (issues: #10741, #11264, #13176, #13978, #4317, #7297)

Plugin Analysis Kuromoji

• Add nbest options and NumberFilter #17173

Plugin Discovery EC2

• Use DefaultAWSCredentialsProviderChain AWS SDK class for credentials #19561 (issue: #19556)
• Support new Asia Pacific (Mumbai) ap-south-1 AWS region #19112 (issue: #19110)
• Add support for proxy authentication for s3 and ec2 #15293 (issue: #15268)

Plugin Discovery GCE

• Allow _gce_ network when not using discovery gce #15765 (issue: #15724)

Plugin Ingest Attachment

• Minor attachment processor improvements #16574

Plugin Lang Painless

• Disable regexes by default in painless #20427 (issue: #20397)
• Catch OutOfMemory and StackOverflow errors in Painless #19936
• Change Painless Tree Structure for Variable/Method Chains #19459
• Add replaceAll and replaceFirst #19070
• Painless Initializers #19012
• Add augmentation #19003
• Infer lambda arguments/return type #18983
• Fix explicit casts and improve tests. #18958
• Add lambda captures #18954
• improve Debugger to print code even if it hits exception #18932 (issue: #1)
• Move semicolon hack into lexer #18931
• Add flag support to regexes #18927
• improve lambda syntax (allow single expression) #18924
• Remove useless dropArguments in megamorphic cache #18913
• non-capturing lambda support #18911 (issue: #18824)
• fix bugs in operators and more improvements for the dynamic case #18899
• improve unary operators and cleanup tests #18867 (issue: #18849)
• Add support for the find operator (=) and the match operator (==) #18858
• Remove casts and boxing for dynamic math #18849 (issue: #18847)
• Refactor def math #18847
• Add support for /regex/ #18842
• Array constructor references #18831
• Method references to user functions #18828
• Add } as a delimiter. #18827 (issue: #18821)
• Add Lambda Stub Node #18824
• Add capturing method references #18818 (issue: #18748)
• Add Functions to Painless #18810
• Add Method to Get New MethodWriters #18771
• Static For Each #18757
• Method reference support #18748 (issue: #18578)
• Add support for the new Java 9 MethodHandles#arrayLength() factory #18734
• Improve painless compile-time exceptions #18711 (issue: #18600)
• add java.time packages to painless whitelist #18621
• Add Function Reference Stub to Painless #18578
• improve painless whitelist coverage of java api #18533
• Definition cleanup #18463
• Made def variable casting consistent with invokedynamic rules #18425
• Use Java 9 Indy String Concats, if available #18400 (issue: #18398)
• Add method overloading based on arity #18385
• Refactor WriterUtils to extend ASM GeneratorAdapter #18382
• Whitelist expansion #18372
• Remove boxing when loading and storing values in "def" fields/arrays, remove boxing onsimple method calls of "def" methods #18359
• Some cleanups #18352
• Use isAssignableFrom instead of relying on ClassCastException #18350
• Build descriptor of array and field load/store in code #18338
• Rename the dynamic call site factory to DefBootstrap #18335
• Cleanup of DynamicCallSite #18323
• Improve exception stacktraces #18319
• Make Line Number Available in Painless #18298
• Remove input, support params instead #18287
• Decouple ANTLR AST from Painless #18286
• _value support in painess? #18284
• Long priority over Float #18282
• _score as double, not float #18277
• Add ctx keyword to painless. #18264
• Painless doc access #18262
• Retrieve _score directly from Scorer #18258
• Implement needsScore() correctly. #18247
• Add synthetic length property as alias to Lists, so they can be used like arrays #18241
• Use better typing for dynamic method calls #18234
• Array load/store and length with invokedynamic #18232 (issue: #18201)
• Switch painless dynamic calls to invokedynamic, remove perf hack/cheat #18201
• Add fielddata accessors (.value/.values/.distance()/etc) #18169
• painless: optimize/simplify dynamic field and method access #18151
• Painless: Single-Quoted Strings #18150
• Painless Clean Up #17428
• Make Painless a Module #16755
• Minor Clean up #16457
• Remove Extra String Concat Token #16382

Plugin Mapper Attachment

• minor attachments cleanups: IDE test support and EPUB format #14626

Plugin Mapper Size

• Add doc values support to the _size field in the mapper-size plugin #19217 (issue: #18334)

Plugin Repository Azure

• Support global repositories.azure. settings #15141 (issue: #13776)
• Add timeout settings (default to 5 minutes) #15080 (issue: #14277)
• Remove AbstractLegacyBlobContainer #14650 (issue: #13434)

Plugin Repository HDFS

• merge current hdfs improvements to master #15588

Plugin Repository S3

• Extract AWS Key from KeyChain instead of using potential null value #19560 (issues: #18703, #19557)
• Check that S3 setting buffer_size is always lower than chunk_size #17274 (issue: #17244)

Plugins

• Revert "Display plugins versions" #20807 (issues: #18683, #20668)
• Provide error message when plugin id is missing #20660
• Print message when removing plugin with config #20338
• Plugins: Update official plugin location with unified release #19996
• Allow plugins to upgrade global custom metadata on startup #19962
• Switch aggregations from push to pull #19839
• Display plugins versions #18683
• Add ScriptService to dependencies available for plugin components #19770
• Make NamedWriteableRegistry immutable and add extension point for named writeables #19764
• Log one plugin info per line #19441
• Make rest headers registration pull based #19440
• Add resource watcher to services available for plugin components #19401
• Add some basic services to createComponents for plugins #19380
• Make plugins closeable #19137
• Plugins: Add status bar on download #18695
• Add did-you-mean for plugin cli #18942 (issue: #18896)
• Plugins: Remove name() and description() from api #18906
• Emit nicer error message when trying to install unknown plugin #18876 (issue: #17226)
• Add plugin information for Verbose mode #18051 (issue: #16375)
• Cli: Improve output for usage errors #17938
• Cli: Add verbose output with zip url when installing plugin #17662 (issue: #17529)
• PluginManager: Add xpack as official plugin #17227
• CliTool: Cleanup and document Terminal #16443
• Plugin cli: Improve maven coordinates detection #16384 (issue: #16376)
• Enforce plugin zip does not contain zip entries outside of the plugin dir #16361
• CliTool: Allow unexpected exceptions to propagate #16359
• Reduce complexity of plugin cli #16336
• Remove Plugin.onIndexService. #15029 (issue: #14896)
• Open up QueryCache and SearcherWrapper extension points #14303

Query DSL

• Throw exception when multiple field names are provided as part of query short syntax #19871 (issue: #19791)
• Query parsers to throw exception when multiple field names are provided #19791 (issue: #19547)
• Allow empty json object in request body in _count API #19595 (issue: #19422)
• Treat zero token in common terms query as MatchNoDocsQuery #18656
• Handle empty query bodies at parse time and remove EmptyQueryBuilder #17624 (issues: #17540, #17541)
• Enforce MatchQueryBuilder#maxExpansions() to be strictly positive #18464
• Don’t allow fuzziness for multi_match types cross_fields, phrase and phrase_prefix #18322 (issues: #6866, #7764)
• Add MatchNoDocsQuery, a query that matches no documents and prints the reason why in the toString method. #17780
• Adds ignore_unmapped option to geo queries #17751
• Adds ignore_unmapped option to nested and P/C queries #17748
• SimpleQueryParser should call MappedFieldType.termQuery when appropriate. #17678
• An exists query on an object should query a single term. #17186 (issue: #17131)
• Function Score Query: make parsing stricter #16617 (issue: #16583)
• Parsers should throw exception on unknown objects #14255 (issue: #10974)
• UNICODE_CHARACTER_CLASS fix #11598 (issue: #10146)

Query Refactoring

• Add infrastructure to rewrite query builders #16599
• Switch geo validation to enum #13672 (issue: #13608)

REST

• Add a REST spec for the create API #20924
• Add response params to REST params did you mean #20753 (issues: #20722, #20747)
• Add did you mean to strict REST params #20747 (issue: #20722)
• Add exclusion support to response filtering #19865
• Only write forced_refresh if we forced a refresh #19669 (issue: #19629)
• Add Location header to the index, update, and create APIs #19509 (issue: #19079)
• Add support for wait_for_events to the _cluster/health REST endpoint #19432 (issue: #19419)
• Rename Search Template REST spec names #19178
• Adding status field in _msearch error request bodies #18586 (issue: #18013)
• Add semicolon query string parameter delimiter #18186 (issue: #18175)
• Enable HTTP compression by default with compression level 3 #18066 (issue: #7309)
• Allow JSON with unquoted field names by enabling system property #17801 (issue: #17674)
• More robust handling of CORS HTTP Access Control #16092
• Add option to exclude based on paths in XContent #16017

Recovery

• Pass on maxUnsafeAutoIdTimestamp on recovery / relocation #20300
• Non-blocking primary relocation hand-off #19013 (issues: #15900, #18553)
• index shard should be able to cancel check index on close. #18839 (issue: #12011)
• TransportNodesListGatewayStartedShards should fall back to disk based index metadata if not found in cluster state #17663 (issue: #17630)
• Recover broken IndexMetaData as closed #17187
• Relocation source should be marked as relocating before starting recovery to primary relocation target #16500
• Operation counter for IndexShard #15956 (issue: #15900)
• Primary relocation handoff #15900 (issues: #15532, #16274, #19013)
• Remove recovery threadpools and throttle outgoing recoveries on the master #15372
• Refactor StoreRecoveryService to be a simple package private util class #13766

Reindex API

• Only ask for _version we need it #19693 (issue: #19135)
• Use fewer threads when reindexing-from-remote #19636
• Support authentication with reindex-from-remote #19310
• Support requests_per_second=-1 to mean no throttling in reindex #19101 (issue: #19089)
• Implement ctx.op = "delete" on _update_by_query and _reindex #18614 (issue: #18043)
• Make Reindex cancellation tests more uniform #18498
• Makes DeleteByQueryRequest implements IndicesRequest #18466
• Teach reindex to retry on search failures #18331 (issue: #18059)
• Remove ReindexResponse in favor of BulkIndexByScrollResponse #18205
• Stricter validation of Reindex’s requests_per_second #18028
• Properly mark reindex’s child tasks as child tasks #17770
• Make reindex throttling dynamic #17262
• Throttling support for reindex #17039
• Add ingest pipeline support to reindex #16932

Scripting

• Parse script on storage instead of on retrieval #20356
• Migrate elasticsearch native script examples to the main repo #19334 (issue: #14662)
• Remove ClusterState from compile api #19136
• Mustache: Render Map as JSON #18856 (issue: #18970)
• Compile each Groovy script in its own classloader #18918 (issue: #18572)
• Include script field even if it value is null #18384 (issue: #16408)
• Skipping hidden files compilation for script service #16286 (issue: #15269)
• Rename Plan A to Painless #16245
• Add plumbing for script compile-time parameters #15464
• Factor mustache → modules/lang-mustache #15328

Scroll

• Add an index setting to limit the maximum number of slices allowed in a scroll request. #18782

Search

• Limit batch size when scrolling #19367 (issue: #19249)
• Record method counts while profiling query components #18302
• Change default similarity to BM25 #18948 (issue: #18944)
• Add a parameter to cap the number of searches the msearch api will concurrently execute #18721
• Introduces GeoValidationMethod to GeoDistanceSortBuilder #18036
• Switches from empty boolean query to matchNoDocs #18007 (issue: #17981)
• Allow binary sort values. #17959 (issues: #17971, #6077)
• Fail query if it contains very large rescores #17917 (issue: #17522)
• Type filters should not have a performance impact when there is a single type. #17350
• Store _all payloads on 1 byte instead of 4. #16899
• Refuse to load fields from _source when using the fields option and support wildcards. #15017 (issues: #10783, #14489)
• Add response into ClearScrollResponse #13835 (issue: #13817)
• Shuffle shards for _only_nodes + support multiple specifications like cluster API #12575 (issues: #12546, #12700)

Search Refactoring

• Removes the now obsolete SearchParseElement implementations #18233
• Remove RescoreParseElement #17441
• Remove HighlighterParseElement #17303
• Move top level parsing of sort element to SortBuilder #17248
• Switch to using refactored SortBuilder instead of using BytesReference in serialization #17205 (issues: #17146, #17257)
• Add build() method to SortBuilder implementations #17146 (issue: #10217)
• Refactoring of Suggestions #17096 (issue: #10217)
• Move sort order field up into SortBuilder #17035
• Moves SortParser:parse(…) to only require QueryShardContext #16999 (issue: #15178)
• Change internal representation of suggesters #16873
• Make GeoDistanceSortBuilder serializable, 2nd try #16572 (issues: #15178, #16151)
• Move missing() from SortBuilder interface to class #16225 (issues: #15178, #16151)
• Remove deprecated parameters from ScriptSortBuilder #16153 (issue: #15178)
• Refactor GeoSortBuilder #16151 (issue: #15178)
• Refactor FieldSortBuilder #16127 (issue: #15178)
• Make sort order enum writable. #16124 (issue: #15178)
• Make DistanceUnit writable. #16122 (issue: #15178)
• RescoreBuilder: Add parsing and creating of RescoreSearchContext #16014 (issue: #15559)
• Make RescoreBuilder and nested QueryRescorer Writable #15953 (issue: #15559)
• Explain api: move query parsing to the coordinating node #14270
• Switch query parsers to use ParseField #14249 (issue: #8964)
• Refactoring of Aggregations #14136

Sequence IDs

• Persist sequence number checkpoints #18949 (issue: #10708)
• Add sequence numbers to cat shards API #18772

Settings

• Add precise logging on unknown or invalid settings #20951 (issue: #20946)
• Make action.auto_create_index setting a dynamic cluster setting #20274 (issue: #7513)
• Removes space between # and the setting in elasticsearch.yml #20094 (issue: #20090)
• Validates new dynamic settings from the current state #19122 (issue: #19046)
• Improve error message if a setting is not found #18920 (issue: #18663)
• Cleanup placeholder replacement #17335
• Switch to registered Settings for all IndexingMemoryController settings #17778 (issue: #17442)
• Add guard against null-valued settings #17310 (issue: #17292)
• Useful error message for null property placeholder #17293 (issue: #17292)
• Archive cluster level settings if unknown or broken #17246
• Improve error message if setting is not found #17230
• Improve upgrade experience of node level index settings #17223 (issue: #17187)
• Settings with complex matchers should not overlap #16754
• Moves GCE settings to the new infra #16722 (issue: #16720)
• Add filtering support within Setting class #16629 (issue: #16598)
• Migrate AWS settings to new settings infrastructure #16602 (issue: #16293)
• Remove gateway.initial_meta and always rely on min master nodes #16446
• Rewrite SettingsFilter to be immutable #16425
• Simplify azure settings #16363
• Convert PageCacheRecycler settings #16341
• Monitor settings #16313
• Cut over tribe node settings to new settings infra #16311
• Convert multcast plugin settings to the new infra #16295
• Convert request.headers.* to the new settings infra #16292
• Migrate Azure settings to new settings infrastructure #16291
• Validate logger settings and allow them to be reset via API #16289
• Switch NodeEnvironment’s settings to new settings #16273
• Simplify AutoCreateIndex and add more tests #16270
• Convert several pending settings #16269
• Migrate query caching settings to the new settings infra. #16267
• Convert action.auto_create_index and action.master.force_local to the new settings infra #16263
• Convert cluster.routing.allocation.type and processors to the new settings infra. #16238
• Validate tribe node settings on startup #16237
• Move node.client, node.data, node.master, node.local and node.mode to new settings infra #16230
• Moved http settings to the new settings infrastructure #16188
• Migrate network service to the new infra #16187
• Convert client.transport settings to new infra #16183
• Move discovery.* settings to new Setting infrastructure #16182
• Change over to o.e.common.settings.Setting for http settings #16181
• Convert "path.*" and "pidfile" to new settings infra #16180
• Migrate repository settings to the new settings API #16178
• Convert "indices.*" settings to new infra. #16177
• Migrate gateway settings to the new settings API. #16175
• Convert several node and test level settings #16172
• Run Metadata upgrade tool on every version #16168
• Check for invalid index settings on metadata upgrade #16156
• Validate the settings key if it’s simple chars separated by . #16120
• Validate known global settings on startup #16091
• Cut over all index scope settings to the new setting infrastrucuture #16054 (issues: #12790, #12854, #16032, #6732)
• Remove updatability of index.flush_on_close #15964 (issue: #15955)
• Move all dynamic settings and their config classes to the index level #15955 (issue: #6732)
• Always require units for bytes and time settings #15948 (issue: #11437)
• Make MetaData parsing less lenient. #15828
• Move async translog sync logic into IndexService #15584
• Remove index.merge.scheduler.notify_on_failure and default to true #15572 (issue: #15570)
• Remove cache concurrency level settings that no longer apply #14210 (issues: #13224, #13717, #7836)

Similarities

• Defining a global default similarity #16682 (issue: #16594)

Snapshot/Restore

• Delete differing files in the store before restoring #20220 (issue: #20148)
• Adds ignoreUnavailable option to the snapshot status API #20066 (issue: #18522)
• Check restores in progress before deleting a snapshot #19853
• Snapshot repository cleans up empty index folders #19751
• BlobContainer#writeBlob no longer can overwrite a blob #19749 (issue: #15579)
• More resilient blob handling in snapshot repositories #19706 (issues: #18156, #18815, #19421, #7540)
• Adding repository index generational files #19002 (issue: #18156)
• Raised IOException on deleteBlob #18815 (issue: #18530)
• Adds UUIDs to snapshots #18228 (issue: #18156)
• Clarify the semantics of the BlobContainer interface #18157 (issue: #15580)
• Change BlobPath.buildAsString() method #18461
• Remove the Snapshot class in favor of using SnapshotInfo #18167 (issue: #18156)

Stats

• Add mem section back to cluster stats #20255 (issue: #17278)
• Add network types to cluster stats #20144
• Add missing field type in the FieldStats response. #19241 (issue: #17750)
• Expose the ClusterInfo object in the allocation explain output #19106 (issue: #14405)
• Add total_indexing_buffer/_in_bytes to nodes info API #18914 (issue: #18651)
• Allow FieldStatsRequest to disable cache #18900
• Remove index_writer_max_memory stat from segment stats #18651 (issues: #14121, #7440)
• Move DocStats under Engine to get more accurate numbers #18587
• Do not return fieldstats information for fields that exist in the mapping but not in the index. #18212 (issue: #17980)
• Add whether the shard state fetch is pending to the allocation explain API #18119 (issue: #17372)
• Add Failure Details to every NodesResponse #17964 (issue: #3740)
• Add I/O statistics on Linux #15915 (issue: #15296)
• Add points to SegmentStats. #17775 (issue: #16974)
• Remove FieldStats.Float. #17749
• Show configured and remaining delay for an unassigned shard. #17515 (issue: #17372)
• indexing stats now contain indexing ops from recovery [ISSUE] #17412
• Normalize unavailable load average #16061 (issues: #12049, #14741, #15907, #15932, #15934)
• Add load averages to OS stats on FreeBSD #15934 (issue: #15917)
• Expose pending cluster state queue size in node stats #14040 (issue: #13610)

Store

• Use mmapfs by default. #17616 (issue: #16983)
• Remove support for legacy checksums #16931
• Rename index folder to index_uuid #16442 (issues: #13264, #13265, #14512, #14932, #15853)

Suggesters

• Move SuggestUtils methods to their respective caller classes #19914 (issue: #19906)
• Remove payload option from completion suggester #19877 (issue: #19536)
• Add bwc support for reading pre-5.0 completion index #17602

Task Manager

• Rename Task Persistence into Storing Task Results #19982
• Fetch result when wait_for_completion #18905
• Create get task API that falls back to the .tasks index #18682
• Add ability to store results for long running tasks #17928
• Move parentTaskId into TransportRequest #17872
• Shorten the serialization of the empty TaskId #17870
• Expose whether a task is cancellable in the _tasks list API #17464 (issue: #17369)
• Add ability to group tasks by common parent #17341
• Add start time and duration to tasks #16829
• Combine node name and task id into single string task id #16744
• Add task status #16356 (issue: #16344)
• Extend tracking of parent tasks to master node, replication and broadcast actions #15931

Translog

• Fsync documents in an async fashion #20145
• Add checksumming and versions to the Translog’s Checkpoint files #19797
• Beef up Translog testing with random channel exceptions #18997
• Do not replay into translog on local recovery #18547
• FSync translog outside of the writers global lock #18360
• Remove ChannelReference and simplify Views #15898
• Simplify TranslogWriter to always write to a stream #15771
• Remove TranslogService and fold it into synchronous IndexShard API #13707

Bug fixes

Aggregations

• Fixed writeable name from range to geo_distance #20860
• Fix date_range aggregation to not cache if now is used #20740
• The top_hits aggregation should compile scripts only once. #20738
• Fix agg profiling when using breadth_first collect mode #20156
• Throw exception when maxBounds greater than minBounds #19855 (issue: #19833)
• Undeprecates aggs in the search request #19674 (issue: #19504)
• Change how nested and reverse_nested aggs know about their nested depth level #19550 (issues: #11749, #12410)
• Make ExtendedBounds immutable #19490 (issue: #19481)
• Aggregations fix: support include/exclude strings for IP and dates #18408 (issue: #17705)
• Fix xcontent rendering of ip terms aggs. #18003 (issue: #17971)
• Improving parsing of sigma param for Extended Stats Bucket Aggregation #17562 (issue: #17499)
• Fixes NPE when no window is specified in moving average request #17556 (issue: #17516)
• Fixes Filter and FiltersAggregation to work with empty query #17542 (issue: #17518)
• Fixes the defaults for keyed in the percentiles aggregations #17217
• Correct typo in class name of StatsAggregator #15264 (issue: #14730)

Allocation

• Keep a shadow replicas' allocation id when it is promoted to primary #20863 (issue: #20650)
• IndicesClusterStateService should clean local started when re-assigns an initializing shard with the same aid #20687
• IndexRoutingTable.initializeEmpty shouldn’t override supplied primary RecoverySource #20638 (issue: #20637)
• Update incoming recoveries stats when shadow replica is reinitialized #20612
• index.routing.allocation.initial_recovery limits replica allocation #20589
• Upon being elected as master, prefer joins' node info to existing cluster state #19743
• Fix NPE when initializing replica shard has no UnassignedInfo #19491 (issue: #19488)
• Make shard store fetch less dependent on the current cluster state, both on master and non data nodes #19044 (issue: #18938)
• Fix recovery throttling to properly handle relocating non-primary shards #18701 (issue: #18640)
• Replica shards must be failed before primary shards #15686

Analysis

• Named analyzer should close the analyzer that it wraps #20197
• Can load non-PreBuiltTokenFilter in Analyze API #20396
• Fix analyzer alias processing #19506 (issue: #19163)

Bulk

• Add not-null precondition check in BulkRequest #18347 (issue: #12038)

CAT API

• Fixes cat tasks operation in detailed mode #19759 (issue: #19755)
• Add index pattern wildcards support to _cat/shards #19655 (issue: #19634)

CRUD

• GET operations should not extract fields from _source. #20158 (issues: #15017, #20102)
• Squash a race condition in RefreshListeners #18806
• Prevent TransportReplicationAction to route request based on stale local routing table #16274 (issues: #12573, #12574)
• Resolves the conflict between alias routing and parent routing by applying the alias routing and ignoring the parent routing. #15371 (issue: #3068)

Cache

• Prevent requests that use scripts or now() from being cached #20750 (issue: #20645)
• Serialize index boost and phrase suggest collation keys in a consistent order #20081 (issue: #19986)

Circuit Breakers

• Never trip circuit breaker in liveness request #18627 (issue: #17951)
• Free bytes reserved on request breaker #18204 (issue: #18144)

Cluster

• Fixes issue with dangling index being deleted instead of re-imported #19666
• Allow routing table to be filtered by index pattern #19688
• Use executor’s describeTasks method to log task information in cluster service #19531
• Acknowledge index deletion requests based on standard cluster state acknowledgment #18602 (issues: #16442, #18558)
• Dangling indices are not imported if a tombstone for the index exists #18250 (issue: #18249)
• Fix issue with tombstones matching active indices in cluster state #18058 (issue: #18054)
• Shard state action channel exceptions #16057 (issue: #15748)

Core

• Makes m case sensitive in TimeValue #19649 (issue: #19619)
• Guard against negative result from FileStore.getUsableSpace when picking data path for a new shard #19554
• Handle rejected execution exception on reschedule #19505
• Fix concurrency bug in IMC that could cause it to check too infrequently #18357
• Iterables.flatten should not pre-cache the first iterator #18355 (issue: #18353)
• Avoid race while retiring executors #18333
• Refactor UUID-generating methods out of Strings #17837 (issue: #17819)
• Node names cleanup #17723 (issue: #17718)
• NullPointerException from IndexingMemoryController when a version conflict happens during recovery #17569
• Handle RejectedExecution gracefully in TransportService during shutdown #16965

Discovery

• Update discovery nodes after cluster state is published #20409
• Add current cluster state version to zen pings and use them in master election #20384 (issue: #20348)

Engine

• Take refresh IOExceptions into account when catching ACE in InternalEngine #20546 (issue: #19975)
• Don’t suppress AlreadyClosedException #19975 (issue: #19861)

Expressions

• replace ScriptException with a better one #18600

Geo

• Incomplete results when using geo_distance for large distances [ISSUE] #17578
• Fix multi-field support for GeoPoint types #15702 (issue: #15701)
• Enforce distance in distance query is > 0 [ISSUE] #15135

Highlighting

• Enable BoostingQuery with FVH highlighter #19984 (issue: #19985)

Index APIs

• Fixes active shard count check in the case of all shards #19760
• Add zero-padding to auto-generated rollover index name increment #19610 (issue: #19484)

Ingest

• no null values in ingest configuration error messages #20616
• JSON Processor was not properly added #20613
• Don’t rebuild pipeline on every cluster state update #20189
• Add dotexpander processor #20078
• Fix NPE when simulating a pipeline with no id #19650
• Change foreach processor to use ingest metadata for array element #19609 (issue: #19592)
• No other processors should be executed after on_failure is called #19545
• rethrow script compilation exceptions into ingest configuration exceptions #19318
• Rename from ingest-useragent plugin to ingest-user-agent and its processor from useragent to user_agent #19261
• Fix ignore_failure behavior in _simulate?verbose and more cleanup #18987
• Pipeline Stats: Fix concurrent modification exception #18177 (issue: #18126)
• Validate properties values according to database type #17940 (issue: #17683)
• Ingest does not close its factories #17626 (issue: #17625)
• Handle regex parsing errors in Gsub and Grok Processors #17260
• add on_failure exception metadata to ingest document for verbose simulate #16562
• The IngestDocument copy constructor should make a deep copy #16248 (issue: #16246)

Inner Hits

• Ensure that that InnerHitBuilder uses rewritten queries #19360 (issue: #19353)

Internal

• Prevent AbstractArrays from release bytes more than once #20819
• IndicesAliasesRequest should not implement CompositeIndicesRequest #20726
• Ensure elasticsearch doesn’t start with unuspported indices #20514 (issue: #20512)
• Remove ListTasksResponse#setDiscoveryNodes() #19773 (issue: #19772)
• Priority values should be unmodifiable #19447
• Extract AbstractBytesReferenceTestCase #19141
• Add XPointValues #18011 (issue: #18010)
• Fix BulkItemResponse.Failure.toString #17871
• Enable unmap hack for java 9 #16986 (issue: #1)
• Fix issues with failed cache loads #14315
• Allow parser to move on the START_OBJECT token when parsing search source #14145
• Ensure searcher is release if wrapping fails #14107
• Avoid deadlocks in Cache#computeIfAbsent #14091 (issue: #14090)

Java API

• fix IndexResponse#toString to print out shards info #20562
• Add NamedWriteables from plugins to TransportClient #19825 (issue: #19764)
• Fix potential NPE in SearchSourceBuilder #16905 (issue: #16902)

Java REST Client

• Rest Client: add slash to log line when missing between host and uri #19325 (issue: #19314)
• Rest Client: HostsSniffer to set http as default scheme #19306

Logging

• Fix logger when you can not create an azure storage client #20670 (issues: #20633, #20669)
• Avoid unnecessary creation of prefix loggers #20571 (issue: #20570)
• Fix logging hierarchy configs #20463
• Fix prefix logging #20429
• Hack around Log4j bug rendering exceptions #20306 (issue: #20304)
• Avoid prematurely triggering logger initialization #20170
• Only log running out of slots when out of slots #19637

Mapping

• Allow position_gap_increment for fields in indices created prior to 5.0 #20806 (issues: #19510, #20413)
• Validate blank field name #19860 (issue: #19251)
• Better error message when mapping configures null #18809 (issue: #18803)
• Make doc_values accessible for _type #18220
• Fix and test handling of null_value. #18090 (issue: #18085)
• Fail automatic string upgrade if the value of index is not recognized. #18082 (issue: #18062)
• Fix dynamic check to properly handle parents #17864 (issues: #17644, #17854)
• Fix array parsing to remove its context when finished parsing #17768
• Disallow fielddata loading on text fields that are not indexed. #17747
• Make dynamic template parsing less lenient. #17249
• Fix dynamic mapper when its parent already has an update #17065
• Fix copy_to when the target is a dynamic object field. #15216 (issues: #111237, #11237)
• Preserve existing mappings on batch mapping updates #15130 (issues: #14899, #15129)

Network

• Fix connection close header handling #20956 (issue: #20938)
• Ensure port range is readable in the exception message #20893
• Fix expect 100 continue header handling #19904 (issue: #19834)
• Fixes netty4 module’s CORS config to use defaults #19874
• Fix various concurrency issues in transport #19675
• Verify lower level transport exceptions don’t bubble up on disconnects #19518 (issue: #19096)

Packaging

• [Packaging] Do not remove scripts directory on upgrade #20452
• [Package] Remove bin/lib/modules directories on RPM uninstall/upgrade #20448
• Fix handling of spaces for jvm.options on Windows #19951 (issue: #19941)
• Disable service in pre-uninstall #19328
• Remove extra bin/ directory in bin folder #18630
• Filter client/server VM options from jvm.options #18473
• Preserve config files from RPM install #18188 (issue: #18158)
• Fix typo in message for variable setup ES_MAX_MEM #18168
• Don’t run mkdir when $DATA_DIR contains a comma-separated list #17419 (issue: #16992)
• Fix exit code #17082
• Set MAX_OPEN_FILES to 65536 #17431 (issue: #17430)
• [windows] Service command still had positional start command #17391
• Do not pass double-dash arguments on startup #17087 (issue: #17084)

Parent/Child

• Make sure that no _parent#null gets introduces as default _parent mapping #19470 (issue: #19389)

Percolator

• Fail indexing percolator queries containing either a has_child or has_parent query #20229 (issue: #2960)
• Add support for MatchNoDocsQuery in percolator’s query terms extract service #18492
• Let PercolatorQuery’s explain use the two phase iterator #17315 (issue: #17314)

Plugin Discovery Azure Classic

• Make discovery-azure plugin work again #19062 (issues: #15630, #18637)

Plugin Discovery EC2

• Fix EC2 discovery settings #18690 (issues: #18652, #18662)
• Add TAG_SETTING to list of allowed tags for the ec2 discovery plugin. #18257
• Fix EC2 Discovery settings #17651 (issue: #16602)

Plugin Discovery GCE

• Fix NPE when GCE region is empty #19176 (issue: #16967)

Plugin Ingest Attachment

• Adds content-length as number #19927 (issue: #19924)

Plugin Ingest GeoIp

• [ingest-geoip] update geoip to not include null-valued results from #20455

Plugin Lang Painless

• Fix String Concatenation Bug In Painless #20623
• Fix break bug in for/foreach loops. #20146
• Fix compound assignment with string concats #18933 (issue: #18929)
• Fix horrible capture #18907 (issue: #18899)
• Fix Casting Bug #18871
• Remove Grammar Ambiguities #18531
• Remove if/else ANTLR ambiguity. #18428
• Fix insanely slow compilation #18410 (issue: #18398)
• Fix Bug in Painless Assignment #18379
• Fix bracket shortcuts #18263

Plugin Repository Azure

• Register group setting for repository-azure accounts #19086
• Fix azure files removal #18451 (issues: #16472, #18436)

Plugin Repository S3

• Fixes leading forward slash in S3 repository base_path #20861
• Add missing permission to repository-s3 #19128 (issues: #18539, #19121)
• Fix repository S3 Settings and add more tests #18703 (issues: #18662, #18690)

Plugin Store SMB

• Fix calling ensureOpen() on the wrong directory (master forwardport) #16395 (issue: #16383)

Plugins

• Use sysprop like with es.path.home to pass conf dir #18870 (issue: #18689)
• Quote path to java binary #17496 (issue: #17495)
• CliTool: Messages printed in Terminal should have percent char escaped #16367

Query DSL

• Fixes MultiMatchQuery so that it doesn’t provide a null context #20882
• Fix silently accepting malformed queries #20515 (issue: #20500)
• query_string_query should take term length into consideration when fuzziness is auto #20299 (issue: #15972)
• Throw ParsingException if a query is wrapped in an array #19750 (issue: #12887)
• Restore parameter name auto_generate_phrase_queries #19514 (issue: #19512)
• Resolve string dates and date math to millis before evaluating for rewrite in range query #17239
• constant_score query should throw error on more than one filter #17135 (issue: #17126)
• Single IPv4 addresses in IP field term queries #16068 (issue: #16058)
• Make strategy optional in GeoShapeQueryBuilder readFrom and writeTo #13963

Query Refactoring

• Query refactoring: set has_parent & has_child types context properly #13863
• Make sure equivalent geohashCellQueries are equal after toQuery called #13792

REST

• Remove lenient URL parameter parsing #20722 (issue: #14719)
• Fixes CORS handling so that it uses the defaults #19522 (issue: #19520)
• Get XContent params from request in Nodes rest actions #18860 (issue: #18794)
• Fixes reading of CORS pre-flight headers and methods #17523 (issue: #17483)

Recovery

• Fix concurrency issues between cancelling a relocation and marking shard as relocated #20443
• Move reset recovery into RecoveriesCollection #19466 (issue: #19473)
• Fix replica-primary inconsistencies when indexing during primary relocation with ongoing replica recoveries #19287 (issue: #19248)
• Invoke IndexingOperationListeners also when recovering from store or remote #17406
• Prevent interruption while store checks lucene files for consistency #16308
• Mark shard as recovering on the cluster state thread #14276 (issues: #13766, #14115)

Reindex API

• Fix reindex with transport client #19997 (issues: #19773, #19979)
• Fix a race condition in reindex’s rethrottle #18731 (issue: #18744)
• Reindex should never report negative throttled_until #17799 (issue: #17783)
• Reindex should gracefully handle when _source is disabled #17667 (issue: #17666)

Scripting

• Add support for booleans in scripts #20950 (issue: #20949)
• Fix Javascript OOM build Failure #20307
• Fix propagating the default value for script settings #20183 (issue: #20159)
• Catch and wrap AssertionError and NoClassDefFoundError in groovy scripts #19958 (issues: #19806, #19923)

Search

• Do not cache script queries. #20799 (issue: #20763)
• Throw error when trying to fetch fields from source and source is disabled #20424 (issues: #20093, #20408)
• Source filtering should keep working when the source contains numbers greater than Long.MAX_VALUE. #20278 (issue: #11508)
• Fix NPE when running a range query on a scaled_float with no upper bound. #20253
• Fix NPE during search with source filtering if the source is disabled. #20093 (issue: #7758)
• Restore assignment of time value when deserializing a scroll instance #19977 (issue: #18820)
• Fix explain output for dfs query #19972 (issue: #15369)
• Don’t recursively count children profile timings #19397 (issue: #18693)
• fix explain in function_score if no function filter matches #19185
• Fix NPEs due to disabled source #18957
• Require timeout units when parsing query body #19077 (issue: #19075)
• Close SearchContext if query rewrite failed #18727
• Fix parsing single rescore element in SearchSourceBuilder #18440 (issue: #18439)
• Fail queries on not indexed fields. #18014
• Fix for search after #16271
• Do not be lenient when parsing CIDRs #14874 (issue: #14862)

Settings

• Fix Setting.timeValue() method #20696 (issue: #20662)
• Add a hard limit for index.number_of_shard #20682
• Include complex settings in settings requests #20622
• Fix filter cache setting to allow percentages #20335 (issue: #20330)
• Move cluster.routing.allocation.same_shard.host setting to new settings infrastructure #20046 (issue: #20045)
• Validate settings against dynamic updaters on the master #19088 (issue: #19046)
• Register "cloud.node.auto_attributes" setting in EC2 discovery plugin #18678
• Use object equality to compare versions in IndexSettings #18103
• fix exists method for list settings when using numbered setting format #17949
• convert settings for ResourceWatcherService to new infrastructure #17948
• Register bootstrap settings #16513
• Add settings filtering to node info requests #16445
• Ban write access to system properties #14914

Snapshot/Restore

• Better handling of an empty shard’s segments_N file #18784 (issue: #18707)
• Fix race condition in snapshot initialization #18426 (issue: #18121)
• Fix the semantics for the BlobContainer interface #17878 (issues: #15579, #15580)

Stats

• Fix FieldStats deserialization of ip field #20522 (issue: #20516)
• Fix serialization bug in allocation explain API. #19494
• Allocation explain: Also serialize includeDiskInfo field #19492
• Add missing builder.endObject() in FsInfo #18443 (issues: #15915, #18433)

Store

• Tighten up concurrent store metadata listing and engine writes #19684 (issue: #19416)
• Make static Store access shard lock aware #19416 (issue: #18938)
• Catch assertion errors on commit and turn it into a real exception #19357 (issue: #19356)

Task Manager

• Shard level tasks in Bulk Action lose reference to their parent tasks #17743
• Take filterNodeIds into consideration while sending task requests to nodes #17081

Term Vectors

• Fix calculation of took time of term vectors request #17817 (issue: #12565)

Translog

• Fix RAM usage estimation of LiveVersionMap. #20123 (issue: #19787)
• Fix translog replay multiple operations same doc #18611 (issues: #18547, #18623)
• Snapshotting and sync could cause a dead lock TranslogWriter #18481 (issues: #1, #18360, #2)
• Move translog recover outside of the engine #17422
• Mark shard active during recovery; push settings after engine finally inits #16250 (issues: #14121, #16209)

Regressions

Highlighting

• Handle SynonymQuery extraction for the FastVectorHighlighter #20829 (issue: #20781)

Upgrades

Core

• Upgrade to Lucene 6.2.0 #20147 (issue: #20092)
• Update to jackson 2.8.1 #18939 (issue: #18076)
• Upgrade to Lucene 6.1.0. #18926
• Upgrade to lucene-6.1.0-snapshot-3a57bea. #18786
• Upgrade to Lucene 6.0.1. #18648 (issues: #17535, #28)
• Upgrade to lucene 6 release #17657
• Upgrade to lucene-6.0.0-f0aa4fc. #17075
• upgrade to lucene 6.0.0-snapshot-bea235f #16964
• Upgrade to Jackson 2.7.1 #16801 (issue: #16294)

Ingest

• Update MaxMind geoip2 version to 2.6 #16837 (issue: #16801)

Internal

• Bump master (3.0-snapshot) to java 8 #13314

Network

• Upgrade to Netty 4.1.5 #20222 (issue: #19786)
• Dependencies: Upgrade to netty 4.1.4 #19689
• Introduce Netty 4 #19526 (issue: #3226)

Packaging

• Upgrade JNA to 4.2.2 and remove optionality #19045 (issue: #13245)

Plugin Discovery EC2

• Update aws sdk to 1.10.69 and add use_throttle_retries repository setting #17784 (issues: #538, #586, #589)

Scripting

• Dependencies: Updates to mustache 0.9.3 #20337

Search Templates

• Update mustache.java to version 0.9.1 #14053 (issue: #13224)

memo-release-notes-5.0.1.md

5.0.1 Release Notes

Also see Breaking changes in 5.0.

Deprecations

CRUD

• Deprecate VersionType.FORCE #21078 (issue: #20995)

Enhancements

Aggregations

• Rescorer should be applied in the TopHits aggregation #20978 (issue: #19317)

Internal

• ShardActiveResponseHandler shouldn’t hold to an entire cluster state #21470 (issue: #21394)

Network

• Fix handler name on message not fully read #21478

Packaging

• Add empty plugins dir for archive distributions #21204 (issue: #20342)
• Make explicit missing settings for Windows service #21200 (issue: #18317)
• Change permissions on config files #20966

Search

• Optimize query with types filter in the URL (t/t/_search) #20979

Bug fixes

Aggregations

• Rewrite Queries/Filter in FilterAggregationBuilder and ensure client usage marks query as non-cachable #21303 (issue: #21301)
• Thread safety for scripted significance heuristics #21113 (issue: #18120)
• ip_range aggregation should accept null bounds. #21043 (issue: #21006)

CAT API

• Consume full_id request parameter early #21270 (issue: #21266)

Cache

• Fix the request cache keys to not hold references to the SearchContext. #21284

Circuit Breakers

• ClusterState publishing shouldn’t trigger circuit breakers #20986 (issues: #20827, #20960)

Core

• Fix ShardInfo#toString #21319
• Protect BytesStreamOutput against overflows of the current number of written bytes. #21174 (issue: #21159)
• Return target index name even if _rollover conditions are not met #21138

Engine

• Retrying replication requests on replica doesn’t call onRetry #21189 (issue: #20211)

Index APIs

• Validate the _rollover target index name early to also fail if dry_run=true #21330 (issue: #21149)

Ingest

• Stored scripts and ingest node configurations should be included into a snapshot #21227 (issue: #21184)

Internal

• Restore thread’s original context before returning to the ThreadPool #21411

Java API

• Fix InternalSearchHit#hasSource to return the proper boolean value #21441 (issue: #21419)
• Null checked for source when calling sourceRef #21431 (issue: #19279)
• ClusterAdminClient.prepareDeletePipeline method should accept pipeline id to delete #21228

Java REST Client

• Rest client: don’t reuse the same HttpAsyncResponseConsumer across multiple retries #21378

Logging

• Assert status logger does not warn on Log4j usage #21339
• Fix logger names for Netty #21223 (issue: #20457)

Packaging

• Set vm.max_map_count on systemd package install #21507
• Export ES_JVM_OPTIONS for SysV init #21445 (issue: #21255)
• Debian: configure start-stop-daemon to not go into background #21343 (issues: #12716, #21300)
• Generate POM files with non-wildcard excludes #21234 (issue: #21170)

Query DSL

• Max score should be updated when a rescorer is used #20977 (issue: #20651)

REST

• The routing query string param is supported by mget but was missing from the rest spec #21357
• fix thread_pool_patterns path variable definition #21332
• ensure the XContentBuilder is always closed in RestBuilderListener #21124

Reindex API

• Bump reindex-from-remote’s buffer to 200mb #21222 (issue: #21185)
• Fix reindex-from-remote for parent/child from <2.0 #21070 (issue: #21044)

Search

• Fixes cachability problems with fetching TTL values when searching #21493 (issue: #21457)
• Remove LateParsingQuery to prevent timestamp access after context is frozen #21328 (issue: #21295)

Snapshot/Restore

• Ensures cleanup of temporary index-* generational blobs during snapshotting #21469 (issue: #21462)
• Fixes get snapshot duplicates when asking for _all #21340 (issue: #21335)

Upgrades

Core

• Upgrade to Lucene 6.2.1 #21207

Dates

• Update Joda Time to version 2.9.5 #21468 (issue: #20911)

memo-release-notes-5.0.2.md

5.0.2 Release Notes

Also see Breaking changes in 5.0.

Enhancements

Core

• Install a security manager on startup #21716

Exceptions

• Add BWC layer for Exceptions #21694 (issue: #21656)

Logging

• Truncate log messages from the end #21609 (issue: #21602)

Scripting

• Wrap VerifyError in ScriptException #21769

Snapshot/Restore

• Abort snapshots on a node that leaves the cluster #21084 (issue: #20876)

Bug fixes

Allocation

• Allow master to assign primary shard to node that has shard store locked during shard state fetching #21656 (issue: #19416)

Cluster

• Remove cluster update task when task times out #21578 (issue: #21568)

Core

• Add a StreamInput#readArraySize method that ensures sane array sizes #21697
• Use a buffer to do character to byte conversion in StreamOutput#writeString #21680 (issue: #21660)

Engine

• Die with dignity on the Lucene layer #21721 (issue: #19272)
• Fix InternalEngine#isThrottled to not always return false. #21592

Index Templates

• Fix integer overflows when dealing with templates. #21628 (issue: #21622)

Ingest

• fix trace_match behavior for when there is only one grok pattern #21413 (issue: #21371)

Internal

• Rethrow ExecutionException from the loader to concurrent callers of Cache#computeIfAbsent #21549
• Fixes potential NullPointerException on shard closing #21515 (issue: #21084)

Java API

• Transport client: Fix remove address to actually work #21743
• Add a HostFailureListener to notify client code if a node got disconnected #21709 (issue: #21424)

Logging

• Do not prematurely shutdown Log4j #21519 (issue: #21514)

Network

• Die with dignity on the network layer #21720 (issue: #19272)

Plugin Lang Painless

• Fix a VerifyError bug in Painless #21765
• Fix Lambdas in Painless to be Able to Use Top-Level Variables Such as params and doc #21635 (issues: #20869, #21479)

Search

• Respect default search timeout #21599 (issues: #12211, #21595)

Settings

• Don’t reset non-dynamic settings unless explicitly requested #21646 (issue: #21593)

Tribe Node

• Add socket permissions for tribe nodes #21546 (issues: #16392, #21122)

memo-release-notes-5.1.0.md

5.1.0 Release Notes

Version 5.1.0 doesn’t exist because, for a short period of time, the Elastic Yum and Apt repositories included unreleased binaries labeled 5.1.0. To avoid confusion and upgrade issues for the people that have installed these without realizing, we decided to skip the 5.1.0 version and release 5.1.1 instead.

memo-release-notes-5.1.1.md

5.1.1 Release Notes

Also see Breaking changes in 5.1.

Breaking changes

Aliases

• Validate alias names the same as index names #20771 (issue: #20748)

REST

• Remove lenient stats parsing 5.x #21576 (issues: #20722, #21410, #21417)

Breaking Java changes

Discovery

• Remove pluggability of ElectMasterService #21031

Exceptions

• Remove IndexTemplateAlreadyExistsException and IndexShardAlreadyExistsException #21539 (issue: #21494)
• Replace IndexAlreadyExistsException with ResourceAlreadyExistsException #21494
• Backport: Replace IndexAlreadyExistsException with ResourceAlreadyExistsException #21601 (issue: #21494)

Internal

• Pass executor name to request interceptor to support async intercept calls #21089

Network

• Unguice Transport and friends #20526

Plugins

• Plugins: Remove support for onModule #21416

Deprecations

Analysis

• Deprecating request parameters of _analyze API in 5.x #20686 (issue: #20246)

CRUD

• Deprecate VersionType.FORCE #21078 (issue: #20995)

Core

• Add deprecation logging for users that explicitly opt in for the default fs type. #21617

Mapping

• Deprecate timestamp and ttl on index requests. #21826 (issue: #21670)

Query DSL

• Add deprecation logging for lenient boolean queries #21570 (issue: #21555)
• Add deprecation logging message for fuzzy query #20993 (issue: #15760)

Search

• Deprecate ignored type parameter in search_shards api #21730 (issue: #21688)

Settings

• Add deprecation logging for the case that store throttling is used. #21618

New features

Analysis

• Expose Lucenes Ukrainian analyzer #21176 (issue: #19433)

CAT API

• Provides a cat api endpoint for templates. #20545 (issue: #20467)

Query DSL

• Add "all fields" execution mode to simple_query_string query #21341 (issues: #19784, #20925)
• Add support for quote_field_suffix to simple_query_string. #21060 (issue: #18641)
• Add "all field" execution mode to query_string query #20925 (issue: #19784)

Reindex API

• Add automatic parallelization support to reindex and friends #20767 (issue: #20624)

Enhancements

Aggregations

• Rescorer should be applied in the TopHits aggregation #20978 (issue: #19317)

Allocation

• Balance step in BalancedShardsAllocator for a single shard #21103
• Process more expensive allocation deciders last #20724 (issue: #12815)
• Separates decision making from decision application in BalancedShardsAllocator #20634
• Split allocator decision making from decision application #20431 (issue: #20347)

Analysis

• Remove AnalysisService and reduce it to a simple name to analyzer mapping #20627 (issues: #19827, #19828)

CAT API

• Adding built-in sorting capability to _cat apis. #20658 (issue: #16975)
• Add health status parameter to cat indices API #20393

Cache

• Do not cache term queries. #21566 (issues: #16031, #20116)
• Parse alias filters on the coordinating node #20916

Cluster

• Cache successful shard deletion checks #21438

Core

• Reduce memory pressure when sending large terms queries. #21776
• Install a security manager on startup #21716
• Log node ID on startup #21673
• Ensure source filtering automatons are only compiled once #20857 (issue: #20839)
• Improve scheduling fairness when batching cluster state changes with equal priority #20775 (issue: #20768)
• Do not log full bootstrap checks exception #19989

Exceptions

• Add BWC layer for Exceptions #21694 (issue: #21656)

Geo

• Optimize geo-distance sorting. #20596 (issue: #20450)

Ingest

• add ignore_missing option to SplitProcessor #20982 (issues: #19995, #20840)

Internal

• Rename ClusterState#lookupPrototypeSafe to lookupPrototype and remove "unsafe" unused variant #21686
• ShardActiveResponseHandler shouldn’t hold to an entire cluster state #21470 (issue: #21394)
• Remove unused ClusterService dependency from SearchPhaseController #21421
• Remove special case in case no action filters are registered #21251
• Use TimveValue instead of long for CacheBuilder methods #20887
• Remove SearchContext#current and all it’s threadlocals #20778 (issue: #19341)

Java REST Client

• Provide error message when rest request path is null #21233 (issue: #21232)

Logging

• Log failure to connect to node at info instead of debug #21809 (issue: #6468)
• Truncate log messages from the end #21609 (issue: #21602)
• Logging shutdown hack #20389 (issue: #20304)
• Disable console logging #20387

Mapping

• Create the QueryShardContext lazily in DocumentMapperParser. #21287

Network

• Grant Netty permission to read system somaxconn #21840
• Lazy resolve unicast hosts #21630 (issues: #14441, #16412)
• Fix handler name on message not fully read #21478
• Handle rejected pings on shutdown gracefully #20842

Packaging

• Add empty plugins dir for archive distributions #21204 (issue: #20342)
• Make explicit missing settings for Windows service #21200 (issue: #18317)
• Change permissions on config files #20966

Plugin Lang Painless

• Add Debug.explain to painless #21723 (issue: #20263)
• Implement the ?: operator in painless #21506
• In painless suggest a long constant if int won’t do #21415 (issue: #21313)
• Support decimal constants with trailing [dD] in painless #21412 (issue: #21116)
• Implement reading from null safe dereferences #21239
• Painless negative offsets #21080 (issue: #20870)

Plugin Repository S3

• Make the default S3 buffer size depend on the available memory. #21299

Plugins

• Clarify that plugins can be closed #21669
• Plugins: Convert custom discovery to pull based plugin #21398
• Removing plugin that isn’t installed shouldn’t trigger usage information #21272 (issue: #21250)
• Remove pluggability of ZenPing #21049
• Make UnicastHostsProvider extension pull based #21036

Query DSL

• Using ObjectParser in MatchAllQueryBuilder and IdsQueryBuilder #21273
• Expose splitOnWhitespace in Query String Query #20965 (issue: #20841)
• Throw error if query element doesn’t end with END_OBJECT #20528 (issue: #20515)
• Remove lowercase_expanded_terms and locale from query-parser options. #20208 (issue: #9978)

Reindex API

• Make reindex-from-remote ignore unknown fields #20591 (issue: #20504)

Scripting

• Wrap VerifyError in ScriptException #21769
• Support binary field type in script values #21484 (issue: #14469)
• Mustache: Add {{#url}}{{/url}} function to URL encode strings #20838
• Expose ctx._now in update scripts #20835 (issue: #17895)

Search

• Add indices and filter information to search shards api output #21738 (issue: #20916)
• remove pointless catch exception in TransportSearchAction #21689
• Optimize query with types filter in the URL (t/t/_search) #20979
• Makes search action cancelable by task management API #20405

Search Templates

• Add profile and explain parameters to template API #20451

Snapshot/Restore

• Abort snapshots on a node that leaves the cluster #21084 (issue: #20876)

Stats

• Remove load average leniency #21380
• Strengthen handling of unavailable cgroup stats #21094 (issue: #21029)
• Add basic cgroup CPU metrics #21029

Task Manager

• Add search task descriptions #21740

Tribe Node

• Add support for merging custom meta data in tribe node #21552 (issues: #20544, #20791, #9372)

Bug fixes

Aggregations

• Rewrite Queries/Filter in FilterAggregationBuilder and ensure client usage marks query as non-cachable #21303 (issue: #21301)
• Percentiles bucket fails for 100th percentile #21218
• Thread safety for scripted significance heuristics #21113 (issue: #18120)

Allocation

• Allow master to assign primary shard to node that has shard store locked during shard state fetching #21656 (issue: #19416)

Analysis

• Can load non-PreBuiltTokenFilter in Analyze API #20396

CAT API

• Consume full_id request parameter early #21270 (issue: #21266)

Cache

• Fix the request cache keys to not hold references to the SearchContext. #21284

Circuit Breakers

• ClusterState publishing shouldn’t trigger circuit breakers #20986 (issues: #20827, #20960)

Cluster

• Remove cluster update task when task times out #21578 (issue: #21568)

Core

• Add a StreamInput#readArraySize method that ensures sane array sizes #21697
• Use a buffer to do character to byte conversion in StreamOutput#writeString #21680 (issue: #21660)
• Fix ShardInfo#toString #21319
• Protect BytesStreamOutput against overflows of the current number of written bytes. #21174 (issue: #21159)
• Return target index name even if _rollover conditions are not met #21138
• .es_temp_file remains after system crash, causing it not to start again #21007 (issue: #20992)

Dates

• Fix time zone rounding edge case for DST overlaps #21550 (issue: #20833)

Engine

• Die with dignity on the Lucene layer #21721 (issue: #19272)
• Fix InternalEngine#isThrottled to not always return false. #21592
• Retrying replication requests on replica doesn’t call onRetry #21189 (issue: #20211)

Highlighting

• Fix FiltersFunctionScoreQuery highlighting #21827
• Fix highlighting on a stored keyword field #21645 (issue: #21636)

Index APIs

• Validate the _rollover target index name early to also fail if dry_run=true #21330 (issue: #21149)

Index Templates

• Fix integer overflows when dealing with templates. #21628 (issue: #21622)

Ingest

• fix trace_match behavior for when there is only one grok pattern #21413 (issue: #21371)
• Stored scripts and ingest node configurations should be included into a snapshot #21227 (issue: #21184)

Inner Hits

• Skip adding a parent field to nested documents. #21522 (issue: #21503)

Internal

• Rethrow ExecutionException from the loader to concurrent callers of Cache#computeIfAbsent #21549
• Restore thread’s original context before returning to the ThreadPool #21411
• Fix NPE in SearchContext.toString() #21069
• Source filtering should treat dots in field names as sub objects. #20736 (issue: #20719)

Java API

• Transport client: Fix remove address to actually work #21743
• Add a HostFailureListener to notify client code if a node got disconnected #21709 (issue: #21424)
• Fix InternalSearchHit#hasSource to return the proper boolean value #21441 (issue: #21419)
• Null checked for source when calling sourceRef #21431 (issue: #19279)
• ClusterAdminClient.prepareDeletePipeline method should accept pipeline id to delete #21228

Java REST Client

• Rest client: don’t reuse the same HttpAsyncResponseConsumer across multiple retries #21378

Logging

• Do not prematurely shutdown Log4j #21519 (issue: #21514)
• Assert status logger does not warn on Log4j usage #21339
• Fix logger names for Netty #21223 (issue: #20457)

Mapping

• Fail to index fields with dots in field names when one of the intermediate objects is nested. #21787 (issue: #21726)
• Uncommitted mapping updates should not efect existing indices #21306 (issue: #21189)

Network

• DiscoveryNode and TransportAddress should preserve host information #21828
• Die with dignity on the network layer #21720 (issue: #19272)
• Prevent double release in TcpTransport if send listener throws an exception #20880

Packaging

• Set vm.max_map_count on systemd package install #21507
• Export ES_JVM_OPTIONS for SysV init #21445 (issue: #21255)
• Debian: configure start-stop-daemon to not go into background #21343 (issues: #12716, #21300)
• Generate POM files with non-wildcard excludes #21234 (issue: #21170)

Plugin Lang Painless

• Test fix for def equals in Painless #21945 (issue: #21801)
• Fix a VerifyError bug in Painless #21765
• Fix Lambdas in Painless to be Able to Use Top-Level Variables Such as params and doc #21635 (issues: #20869, #21479)

Plugin Mapper Attachment

• NPE is raised when defining a non existing type within attachments type #21848

Query DSL

• Fixes date range query using epoch with timezone #21542 (issue: #21501)
• Allow overriding all-field leniency when lenient option is specified #21504 (issues: #20925, #21341)
• Max score should be updated when a rescorer is used #20977 (issue: #20651)

REST

• Strict level parsing for indices stats #21577 (issue: #21024)
• The routing query string param is supported by mget but was missing from the rest spec #21357
• fix thread_pool_patterns path variable definition #21332
• ensure the XContentBuilder is always closed in RestBuilderListener #21124
• XContentBuilder: Avoid building self-referencing objects #20550 (issues: #19475, #20540)

Reindex API

• Ignore IllegalArgumentException with assertVersionSerializable #21409 (issues: #20767, #21350)
• Bump reindex-from-remote’s buffer to 200mb #21222 (issue: #21185)
• Fix reindex-from-remote for parent/child from <2.0 #21070 (issue: #21044)

Search

• Fix match_phrase_prefix on boosted fields #21623 (issue: #21613)
• Respect default search timeout #21599 (issues: #12211, #21595)
• Remove LateParsingQuery to prevent timestamp access after context is frozen #21328 (issue: #21295)

Search Templates

• SearchTemplateRequest to implement CompositeIndicesRequest #21865 (issue: #21747)

Settings

• Handle spaces in action.auto_create_index gracefully #21790 (issue: #21449)
• Fix settings diff generation for affix and group settings #21788
• Don’t reset non-dynamic settings unless explicitly requested #21646 (issue: #21593)

Snapshot/Restore

• Fixes shard level snapshot metadata loading when index-N file is missing #21813
• Ensures cleanup of temporary index-* generational blobs during snapshotting #21469 (issue: #21462)
• Fixes get snapshot duplicates when asking for _all #21340 (issue: #21335)
• Keep snapshot restore state and routing table in sync (5.x backport) #21131 (issue: #20836)

Task Manager

• Task cancellation command should wait for all child nodes to receive cancellation request before returning #21397 (issue: #21126)

Tribe Node

• Add socket permissions for tribe nodes #21546 (issues: #16392, #21122)

Upgrades

Core

• Upgrade to lucene-6.3.0. #21464

Dates

• Update Joda Time to version 2.9.5 #21468 (issues: #20911, #332, #373, #378, #379, #386, #394, #396, #397, #404, #69)

Logging

• Upgrade Log4j 2 to version 2.7 #20805 (issue: #20304)

Network

• Upgrade to Netty 4.1.6 #21051

Plugin Ingest Attachment

• Update to Tika 1.14 #21663 (issues: #20710, #21591)

memo-release-notes-5.1.2.md

5.1.2 Release Notes

Also see Breaking changes in 5.1.

Enhancements

Internal

• Don’t output null source node in RecoveryFailedException #21963

Mapping

• Only update DocumentMapper if field type changes #22165

Network

• Disable the Netty recycler #22452 (issues: #22189, #22360, #22406, #5904)
• Tell Netty not to be unsafe in transport client #22284

Plugins

• Provide helpful error message if a plugin exists #22305 (issue: #22084)
• Add shutdown hook for closing CLI commands #22126 (issue: #22111)

Bug fixes

Aggregations

• Allow terms aggregations on pure boolean scripts. #22201 (issue: #20941)
• IP range masks exclude the maximum address of the range. #22018 (issue: #22005)
• Fix other_bucket on the filters agg to be enabled if a key is set. #21994 (issue: #21951)

Allocation

• Promote shadow replica to primary when initializing primary fails #22021
• Trim in-sync allocations set only when it grows #21976 (issue: #21719)

Analysis

• Pre-built analysis factories do not implement MultiTermAware correctly. #21981

CRUD

• Reject external versioning and explicit version numbers on create #21998

Core

• Support negative numbers in readVLong #22314

Exceptions

• Fixing shard recovery error message to report the number of docs correctly for each node #22515 (issue: #21893)

Index APIs

• Ensure shrunk indices carry over version information from its source #22469 (issue: #22373)

Java REST Client

• Don’t use null charset in RequestLogger #22197 (issue: #22190)

Mapping

• The _all default mapper is not completely configured. #22236
• Fix MapperService.allEnabled(). #22227

Nested Docs

• Fix bug in query builder rewrite that ignores the ignore_unmapped option #22456

Packaging

• Another fix for handling of paths on Windows #22132 (issue: #21921)

Percolator

• Fix NPE in percolator’s now range check for percolator queries with range queries #22356 (issue: #22355)

Plugin Lang Painless

• Update Painless Loop Counter to be Higher #22560 (issue: #22508)

Plugin Repository Azure

• readonly on azure repository must be taken into account #22055 (issues: #22007, #22053)

Recovery

• Fix primary relocation for shadow replicas #22474 (issue: #20300)
• Don’t close store under CancellableThreads #22434 (issue: #22325)
• Use a fresh recovery id when retrying recoveries #22325 (issue: #22043)
• Allow flush/force_merge/upgrade on shard marked as relocated #22078 (issue: #22043)

Reindex API

• Don’t close rest client from its callback #22061 (issue: #22027)

Search

• Improve concurrency of ShardCoreKeyMap. #22316
• Make -0 compare less than +0 consistently. #22173 (issue: #22167)
• FiltersAggregationBuilder: rewriting filter queries, the same way as in FilterAggregationBuilder #22076

Stats

• Implement stats for geo_point and geo_shape field #22391 (issue: #22384)
• Use reader for doc stats #22317 (issue: #22285)
• Avoid NPE in NodeService#stats if HTTP is disabled #22060 (issue: #22058)
• Add support for "include_segment_file_sizes" in indices stats REST handler #21879 (issue: #21878)

Term Vectors

• Fix _termvectors with preference to not hit NPE #21959
• Return correct term statistics when a field is not found in a shard #21922 (issue: #21906)

Regressions

Logging

• Restores the original default format of search slow log #21770 (issue: #21711)

memo-release-notes-5.2.0.md

5.2.0 Release Notes

Also see Breaking changes in 5.2.

Breaking changes

Core

• Add system call filter bootstrap check #21940

Breaking Java changes

Allocation

• Cluster Explain API uses the allocation process to explain shard allocation decisions #22182 (issues: #20347, #20634, #21103, #21662, #21691)

Cluster

• Remove PROTO-based custom cluster state components #22336 (issue: #21868)

Ingest

• Change type of ingest doc meta-data field TIMESTAMP to Date #22234 (issue: #22074)

Internal

• Consolidate the last easy parser construction #22095
• Introduce XContentParser#namedObject #22003

Plugins

• Plugins: Replace Rest filters with RestHandler wrapper #21905

Query DSL

• Resolve index names in indices_boost #21393 (issue: #4756)

Search

• Cluster search shards improvements: expose ShardId, adjust visibility of some members #21752

Deprecations

Java API

• Deprecate and remove "minimumNumberShouldMatch" in BoolQueryBuilder #22403

Shadow Replicas

• Add a deprecation notice to shadow replicas #22647 (issue: #22024)

New features

Analysis

• Synonym Graph Support (LUCENE-6664) #21517

Ingest

• introduce KV Processor in Ingest Node #22272 (issue: #22222)

Mapping

• Add the ability to set a normalizer on keyword fields. #21919 (issue: #18064)
• Add RangeFieldMapper for numeric and date range types #21002 (issue: #20999)

Enhancements

Aggregations

• value_type is useful regardless of scripting. #22160 (issue: #20163)
• Support for partitioning set of terms #21626 (issue: #21487)

Allocation

• Adds setting level to allocation decider explanations #22268 (issue: #21771)
• Improves allocation decider decision explanation messages #21771
• Prepares allocator decision objects for use with the allocation explain API #21691

Analysis

• Expose FlattenGraphTokenFilter #22643

CRUD

• Use correct block levels for TRA subclasses #22224

Cluster

• Keep NodeConnectionsService in sync with current nodes in the cluster state #22509
• Add a generic way of checking version before serializing custom cluster object #22376 (issue: #22313)
• Add validation for supported index version on node join, restore, upgrade & open index #21830 (issue: #21670)
• Let ClusterStateObserver only hold onto state that’s needed for change detection #21631 (issue: #21568)
• Remove mutable status field from cluster state #21379

Core

• Rename bootstrap.seccomp to bootstrap.system_call_filter #22226 (issue: #21940)
• Cleanup random stats serialization code #22223
• Avoid corruption when deserializing booleans #22152
• Reduce memory pressure when sending large terms queries. #21776

Discovery

• Simplify Unicast Zen Ping #22277 (issues: #19370, #21739, #22120, #22194)
• Prefer joining node with conflicting transport address when becoming master #22134 (issues: #22049, #22120)

Ingest

• add ignore_missing flag to ingest plugins #22273
• Added ability to remove pipelines via wildcards (#22149) #22191 (issue: #22149)
• Enables the ability to inject serialized json fields into root of document #22179 (issue: #21898)
• compile ScriptProcessor inline scripts when creating ingest pipelines #21858 (issue: #21842)

Internal

• Remove some more usages of ParseFieldMatcher #22398 (issues: #19552, #22130)
• Remove some more usages of ParseFieldMatcher #22395 (issues: #19552, #22130)
• Remove some ParseFieldMatcher usages #22389 (issues: #19552, #22130)
• Add infrastructure to manage network connections outside of Transport/TransportService #22194
• Replace strict parsing mode with response headers assertions #22130 (issues: #11859, #19552, #20993)
• Start using ObjectParser for aggs. #22048 (issue: #22009)
• Don’t output null source node in RecoveryFailedException #21963
• ClusterService should expose "applied" cluster states (i.e., remove ClusterStateStatus) #21817

Java High Level REST Client

• Add fromxcontent methods to index response #22229
• Add fromXContent() methods for ReplicationResponse #22196 (issue: #22082)
• Add parsing method for ElasticsearchException #22143
• Add fromXContent method to GetResponse #22082

Java REST Client

• Warn log deprecation warnings received from server #21895

Logging

• Log deleting indices at info level #22627 (issue: #22605)
• Log failure to connect to node at info instead of debug #21809 (issue: #6468)

Mapping

• Atomic mapping updates across types #22220
• Only update DocumentMapper if field type changes #22165
• Better error message when _parent isn’t an object #21987

Network

• Disable the Netty recycler #22452 (issues: #22189, #22360, #22406, #5904)
• Tell Netty not to be unsafe in transport client #22284
• Introduce a low level protocol handshake #22094
• Detach handshake from connect to node #22037
• Reduce number of connections per node depending on the nodes role #21849
• Add a connect timeout to the ConnectionProfile to allow per node connect timeouts #21847 (issue: #19719)
• Grant Netty permission to read system somaxconn #21840
• Remove connectToNodeLight and replace it with a connection profile #21799

Packaging

• Improve the out-of-the-box experience #21920 (issues: #18317, #21783)

Plugin Discovery EC2

• Add support for ca-central-1 region to EC2 and S3 plugins #22458 (issue: #22454)
• Support for eu-west-2 (London) cloud-aws plugin #22308 (issue: #22306)
• Add us-east-2 AWS region #21961 (issue: #21881)
• Add setting to set read timeout for EC2 discovery and S3 repository plugins #21956 (issue: #19078)

Plugin Ingest GeoIp

• Cache results of geoip lookups #22231 (issue: #22074)

Plugin Lang Painless

• Improve painless’s ScriptException generation #21762 (issue: #21733)

Plugins

• Pass ThreadContext to transport interceptors to allow header modification #22618 (issue: #22585)
• Provide helpful error message if a plugin exists #22305 (issue: #22084)
• Add shutdown hook for closing CLI commands #22126 (issue: #22111)
• Allow plugins to install bootstrap checks #22110

Query DSL

• Un-deprecate fuzzy query #22088 (issue: #15760)
• support numeric bounds with decimal parts for long/integer/short/byte datatypes #21972 (issue: #21600)

Reindex API

• Reindex: Better error message for pipeline in wrong place #21985
• Timeout improvements for rest client and reindex #21741 (issue: #21707)

Scripting

• Wrap VerifyError in ScriptException #21769
• Log ScriptException’s xcontent if file script compilation fails #21767 (issue: #21733)

Search

• Move SearchTransportService and SearchPhaseController creation outside of TransportSearchAction constructor #21754
• Don’t carry ShardRouting around when not needed in AbstractSearchAsyncAction #21753
• ShardSearchRequest to take ShardId constructor argument rather than the whole ShardRouting #21750
• Use index uuid as key in the alias filter map rather than the index name #21749

Settings

• Speed up filter and prefix settings operations #22249

Snapshot/Restore

• Fixes retrieval of the latest snapshot index blob #22700
• Synchronize snapshot deletions on the cluster state #22313 (issue: #19957)

Stats

• Include unindexed field in FieldStats response #21821 (issue: #21952)

Task Manager

• Improve the error message if task and node isn’t found #22062 (issue: #22027)
• Add descriptions to create snapshot and restore snapshot tasks. #21901 (issue: #21768)
• Add proper descriptions to reindex, update-by-query and delete-by-query tasks. #21841 (issue: #21768)

Bug fixes

Aggregations

• Fix scaled_float numeric type in aggregations #22351 (issue: #22350)
• Allow terms aggregations on pure boolean scripts. #22201 (issue: #20941)
• Fix numeric terms aggregations with includes/excludes and minDocCount=0 #22141 (issue: #22140)
• Fix missing on aggs on boolean fields. #22135 (issue: #22009)
• IP range masks exclude the maximum address of the range. #22018 (issue: #22005)
• Fix other_bucket on the filters agg to be enabled if a key is set. #21994 (issue: #21951)

Allocation

• Cannot force allocate primary to a node where the shard already exists #22031 (issue: #22021)
• Promote shadow replica to primary when initializing primary fails #22021
• Trim in-sync allocations set only when it grows #21976 (issue: #21719)

Analysis

• AsciiFoldingFilter’s multi-term component should never preserve the original token. #21982
• Pre-built analysis factories do not implement MultiTermAware correctly. #21981

CRUD

• Reject external versioning and explicit version numbers on create #21998

Cluster

• IndicesService handles all exceptions during index deletion #22433

Core

• Streamline foreign stored context restore and allow to perserve response headers #22677 (issue: #22647)
• Support negative numbers in readVLong #22314

Discovery

• Close InputStream when receiving cluster state in PublishClusterStateAction #22711
• Do not reply to pings from another cluster #21894 (issue: #21874)

Engine

• Close and flush refresh listeners on shard close #22342

Exceptions

• Fixing shard recovery error message to report the number of docs correctly for each node #22515 (issue: #21893)

Highlighting

• Fix FiltersFunctionScoreQuery highlighting #21827

Index APIs

• Ensure shrunk indices carry over version information from its source #22469 (issue: #22373)

Ingest

• fix index out of bounds error in KV Processor #22288 (issue: #22272)
• Fixes GrokProcessor’s ignorance of named-captures with same name. #22131 (issue: #22117)

Inner Hits

• Inner hits and ignore unmapped #21693 (issue: #21620)

Internal

• Index creation and setting update may not return deprecation logging #22702

Java API

• Don’t output empty ext object in SearchSourceBuilder#toXContent #22093 (issue: #20969)

Java REST Client

• Don’t use null charset in RequestLogger #22197 (issue: #22190)

Mapping

• The _all default mapper is not completely configured. #22236
• Fix MapperService.allEnabled(). #22227
• Dynamic date fields should use the format that was used to detect it is a date. #22174 (issue: #9410)
• Sub-fields should not accept include_in_all parameter #21971 (issue: #21710)
• Mappings: Fix get mapping when no indexes exist to not fail in response generation #21924 (issue: #21916)

Nested Docs

• Fix bug in query builder rewrite that ignores the ignore_unmapped option #22456

Network

• Pass forceExecution flag to transport interceptor #22739
• Ensure new connections won’t be opened if transport is closed or closing #22589 (issue: #22554)
• Prevent open channel leaks if handshake times out or is interrupted #22554
• Execute low level handshake in #openConnection #22440
• Properly configure Netty 3 ClientBootstrap when using custom connection profile #22363 (issue: #22277)
• Handle connection close / reset events gracefully during handshake #22178
• Do not lose host information when pinging #21939 (issue: #21828)
• DiscoveryNode and TransportAddress should preserve host information #21828

Packaging

• Another fix for handling of paths on Windows #22132 (issue: #21921)
• Fix handling of spaces in Windows paths #21921 (issues: #20809, #21525)
• Add option to skip kernel parameters on install #21899 (issue: #21877)

Percolator

• Fix NPE in percolator’s now range check for percolator queries with range queries #22356 (issue: #22355)

Plugin Analysis Stempel

• Fix thread safety of Stempel’s token filter factory #22610 (issue: #21911)

Plugin Lang Painless

• Whitelist some ScriptDocValues in painless #22600 (issue: #22584)
• Update Painless Loop Counter to be Higher #22560 (issue: #22508)
• Fix some issues with painless’s strings #22393 (issue: #22372)
• Test fix for def equals in Painless #21945 (issue: #21801)
• Fix a VerifyError bug in Painless #21765

Plugin Mapper Attachment

• NPE is raised when defining a non existing type within attachments type #21848

Plugin Repository Azure

• Fixes default chunk size for Azure repositories #22577 (issue: #22513)
• readonly on azure repository must be taken into account #22055 (issues: #22007, #22053)

Query DSL

• Fix script score function that combines _score and weight #22713 (issue: #21483)

REST

• Adds percent-encoding for Location headers #21057 (issue: #21016)

Recovery

• Fix primary relocation for shadow replicas #22474 (issue: #20300)
• Don’t close store under CancellableThreads #22434 (issue: #22325)
• Use a fresh recovery id when retrying recoveries #22325 (issue: #22043)
• Allow flush/force_merge/upgrade on shard marked as relocated #22078 (issue: #22043)

Reindex API

• Fix reindex from remote clearing scroll #22525 (issue: #22514)
• Fix source filtering in reindex-from-remote #22514 (issue: #22507)
• Remove content type detection from reindex-from-remote #22504 (issue: #22329)
• Don’t close rest client from its callback #22061 (issue: #22027)
• Keep context during reindex’s retries #21941

Scripting

• Expose ip fields as strings in scripts. #21997 (issue: #21977)

Search

• Improve concurrency of ShardCoreKeyMap. #22316
• Make -0 compare less than +0 consistently. #22173 (issue: #22167)
• Fix boost_mode propagation when the function score query builder is rewritten #22172 (issue: #22138)
• FiltersAggregationBuilder: rewriting filter queries, the same way as in FilterAggregationBuilder #22076
• Fix cross_fields type on multi_match query with synonyms #21638 (issue: #21633)

Search Templates

• SearchTemplateRequest to implement CompositeIndicesRequest #21865 (issue: #21747)

Settings

• Handle spaces in action.auto_create_index gracefully #21790 (issue: #21449)
• Fix settings diff generation for affix and group settings #21788

Snapshot/Restore

• Gracefully handles pre 2.x compressed snapshots #22267
• URLRepository should throw NoSuchFileException to correctly adhere to readBlob contract #22069 (issue: #22004)
• Fixes shard level snapshot metadata loading when index-N file is missing #21813

Stats

• Implement stats for geo_point and geo_shape field #22391 (issue: #22384)
• Use reader for doc stats #22317 (issue: #22285)
• Avoid NPE in NodeService#stats if HTTP is disabled #22060 (issue: #22058)
• Add support for "include_segment_file_sizes" in indices stats REST handler #21879 (issue: #21878)

Term Vectors

• Fix _termvectors with preference to not hit NPE #21959
• Return correct term statistics when a field is not found in a shard #21922 (issue: #21906)

Regressions

Core

• Source filtering: only accept array items if the previous include pattern matches #22593 (issue: #22557)

Logging

• Restores the original default format of search slow log #21770 (issue: #21711)

Search

• Handle specialized term queries in MappedFieldType.extractTerm(Query) #21889 (issue: #21882)

Upgrades

Core

• Upgrade to Lucene 6.4.0 #22724
• Update Jackson to 2.8.6 #22596 (issue: #22266)
• Upgrade to lucene-6.4.0-snapshot-084f7a0. #22413
• Upgrade to lucene-6.4.0-snapshot-ec38570 #21853

Network

• Upgrade to Netty 4.1.7 #22587

memo-release-notes-5.2.1.md

5.2.1 Release Notes

Also see Breaking changes in 5.2.

Deprecations

Geo

• Geo distance range deprecation #22835

Enhancements

Allocation

• Include stale replica shard info when explaining an unassigned primary #22826

Bug fixes

Cache

• Invalidate cached query results if query timed out #22807 (issue: #22789)

Mapping

• Range types causing GetFieldMappingsIndexRequest to fail due to NullPointerException in RangeFieldMapper.doXContentBody when include_defaults=true is on the query string #22925
• Disallow introducing illegal object mappings (double ..) #22891 (issue: #22794)

Plugin Ingest Attachment

• Add missing mime4j library #22799 (issue: #22764)

Plugin Lang Painless

• Don’t allow casting from void to def in painless #22969 (issue: #22908)
• Fix def invoked qualified method refs #22918

Reindex API

• Reindex: do not log when can’t clear old scroll #22942 (issue: #22937)
• Fix reindex-from-remote from <2.0 #22931 (issue: #22893)

Settings

• Expose search.highlight.term_vector_multi_value as a node level setting #22999

Regressions

Core

• Source filtering: only accept array items if the previous include pattern matches #22593 (issue: #22557)

Upgrades

Internal

• Upgrade to Lucene 6.4.1. #22978

memo-release-notes-5.2.2.md

5.2.2 Release Notes

Also see Breaking changes in 5.2.

Bug fixes

Aggregations

• Restore support for the include/pattern syntax. #23140 (issue: #22933)

Cluster

• Don’t set local node on cluster state used for node join validation #23311 (issue: #21830)
• Cluster allocation explain to never return empty response body #23054

Index APIs

• Ensure shrunk indices carry over version information from its source #22469 (issue: #22373)

Internal

• Always restore the ThreadContext for operations delayed due to a block #23349

Java REST Client

• RestClient asynchronous execution should not throw exceptions #23307

Network

• Respect promises on pipelined responses #23317 (issues: #23310, #23322)
• Ensure that releasing listener is called #23310

Plugin Ingest Attachment

• Remove support for Visio and potm files #23214 (issues: #22077, #22079, #22963)

REST

• Properly encode location header #23133 (issues: #21057, #23115)

Stats

• Fix control group pattern #23219 (issue: #23218)

memo-release-notes-5.3.0.md

5.3.0 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.3.

Breaking changes

Query DSL

• Fix name of enabled_position_increments #22895

Search

• Remove DFS_QUERY_AND_FETCH as a search type #22787

Breaking Java changes

CRUD

• Make index and delete operation execute as single bulk item (backport of #21964) #22812 (issue: #21964)

Internal

• Replace SearchExtRegistry with namedObject #22492
• Replace Suggesters with namedObject #22491

Network

• Remove HttpServer and HttpServerAdapter in favor of a simple dispatch method #22636 (issue: #18482)

Plugins

• Deguice rest handlers #22575

Search

• Remove QUERY_AND_FETCH search type #22996

Deprecations

Core

• Deprecate lenient booleans #22716 (issues: #22200, #22696)

Geo

• Geo distance range deprecation #22835

Java API

• Add BulkProcessor methods with XContentType parameter #23078 (issue: #22691)

Mapping

• Deprecate the include_in_all mapping parameter #22972 (issue: #22970)

Network

• Deprecate Netty 3 #23459 (issue: #23411)
• Deprecate Netty 3 #23411

Plugin Repository Azure

• Deprecate auto creation of container for azure repository #22857 (issues: #22843, #22856)

Plugin Repository S3

• S3 Repository: Deprecate specifying region #22848 (issue: #22758)
• S3 Repository: Deprecate auto creation of s3 bucket for repository #22843 (issue: #22761)
• Deprecate specifying credentials through env vars, sys props, and remove profile files #22567 (issues: #21041, #22479)

REST

• Deprecate ldjson support and document ndjson for bulk/msearch #23051 (issues: #23025, #23049)
• Optionally require a valid content type for all rest requests with content #22691 (issue: #19388)

Scripting

• Change Namespace for Stored Script to Only Use Id #22206

New features

Aggregations

• Initial version of an adjacency matrix using the Filters aggregation #22239 (issue: #22169)

CRUD

• Allow an index to be partitioned with custom routing #22274 (issue: #21585)

Highlighting

• Integrate UnifiedHighlighter #21621 (issue: #21376)

Search

• Add federated cross cluster search capabilities #22502 (issue: #21473)
• Add field collapsing for search request #22337 (issue: #21833)

Settings

• Add infrastructure for elasticsearch keystore #22335

Enhancements

Allocation

• Include stale replica shard info when explaining an unassigned primary #22826

Analysis

• Analyze API Position Length Support #22574

CRUD

• Make index and delete operation execute as a single bulk item #21964

Circuit Breakers

• Add used memory amount to CircuitBreakingException message (#22521) #22693 (issue: #22521)

Cluster

• Connect to new nodes concurrently #22984 (issue: #22828)

Core

• Simplify write failure handling (backport of #19105) #22778 (issue: #19105)
• Simplify ElasticsearchException rendering as a XContent #22611
• Remove setLocalNode from ClusterService and TransportService #22608

Engine

• Replace EngineClosedException with AlreadyClosedExcpetion #22631

Index APIs

• Indexing: Add shard id to indexing operation listener #22606
• Better error when can’t auto create index #22488 (issues: #21448, #22435)

Ingest

• Lazy load the geoip databases #23337

Internal

• Improve connection closing in RemoteClusterConnection #22804 (issue: #22803)
• Remove some more usages of ParseFieldMatcher #22437 (issues: #19552, #22130)
• Introduce ToXContentObject interface #22387 (issue: #16347)

Java API

• prevent NPE when trying to uncompress a null BytesReference #22386

Java High Level REST Client

• Add parsing from xContent to MainResponse #22934
• Parse elasticsearch exception’s root causes #22924
• Add parsing method to BytesRestResponse’s error #22873
• Add parsing method for ElasticsearchException.generateThrowableXContent() #22783
• Add fromxcontent methods to delete response #22712 (issue: #22680)
• Add parsing methods for UpdateResponse #22586
• Add parsing from xContent to InternalSearchHit and InternalSearchHits #22429
• Add fromxcontent methods to index response #22229

Java REST Client

• move ignore parameter support from yaml test client to low level rest client #22637
• Support Preemptive Authentication with RestClient #21336

Logging

• Expose logs base path #22625

Mapping

• Improve error message for ipv6 on legacy ip fields #23136 (issue: #23126)

Network

• Keep the pipeline handler queue small initially #23335

Packaging

• Introduce Java version check #23194 (issue: #21102)

Plugin Discovery EC2

• Read ec2 discovery address from aws instance tags #22819 (issues: #22566, #22743)

Plugin Lang Painless

• Generate reference links for painless API #22775
• Painless: Add augmentation to String for base 64 #22665 (issue: #22648)

Plugin Repository S3

• S3 repository: Add named configurations #22762 (issues: #22479, #22520)

Plugins

• Add the ability to define search response listeners in search plugin #22682

Query DSL

• QueryString and SimpleQueryString Graph Support #22541
• Additional Graph Support in Match Query #22503 (issue: #22490)
• RangeQuery WITHIN case now normalises query #22431 (issue: #22412)

Reindex API

• Increase visibility of doExecute so it can be used directly #22614
• Improve error message when reindex-from-remote gets bad json #22536 (issue: #22330)

Scripting

• Expose multi-valued dates to scripts and document painless’s date functions #22875 (issue: #22162)

Search

• Add a setting to disable remote cluster connections on a node #23005
• First step towards separating individual search phases #22802
• Add parsing from xContent to SearchProfileShardResults and nested classes #22649
• ProfileResult and CollectorResult should print machine readable timing information #22638

Settings

• Improve setting deprecation message #23156 (issue: #22849)
• Add secure settings validation on startup #22894
• Allow comma delimited array settings to have a space after each entry #22591 (issue: #22297)
• Allow affix settings to be dynamic / updatable #22526
• Allow affix settings to delegate to actual settings #22523
• Make s3 repository sensitive settings use secure settings #22479

Snapshot/Restore

• Duplicate snapshot name throws InvalidSnapshotNameException #22921 (issue: #18228)
• Use general cluster state batching mechanism for snapshot state updates #22528 (issue: #14899)

Stats

• Add geo_point to FieldStats #21947 (issue: #20707)

Bug fixes

CRUD

• Fix backport executing ops as single item bulk #23083 (issues: #21964, #23069)

Cache

• Invalidate cached query results if query timed out #22807 (issue: #22789)

Cluster

• Don’t set local node on cluster state used for node join validation #23311 (issues: #21830, #3, #4, #6, #9)
• Allow a cluster state applier to create an observer and wait for a better state #23132 (issue: #21817)
• Cluster allocation explain to never return empty response body #23054

Exceptions

• Stop returning "es." internal exception headers as http response headers #22703 (issue: #17593)

Ingest

• Improve missing ingest processor error #23379 (issue: #23392)
• fix date-processor to a new default year for every new pipeline execution #22601 (issue: #22547)

Internal

• Always restore the ThreadContext for operations delayed due to a block #23349
• Fix handling of document failure exception in InternalEngine (backport #22718) #22910

Java REST Client

• RestClient asynchronous execution should not throw exceptions #23307

Mapping

• Switch include_in_all in multifield to warning #23656 (issues: #21971, #23654)
• Fix MapperService StackOverflowError #23605 (issue: #23604)
• Range types causing GetFieldMappingsIndexRequest to fail due to NullPointerException in RangeFieldMapper.doXContentBody when include_defaults=true is on the query string #22925
• Disallow introducing illegal object mappings (double ..) #22891 (issue: #22794)

Network

• Respect promises on pipelined responses #23317 (issues: #23310, #23322)
• Ensure that releasing listener is called #23310

Packaging

• Fall back to non-atomic move when removing plugins #23548 (issue: #35)

Parent/Child

• Add null check in case of orphan child document #22772 (issue: #22770)

Plugin Ingest Attachment

• Remove support for Visio and potm files #23214 (issues: #22077, #22079, #22963)
• Add missing mime4j library #22799 (issue: #22764)

Plugin Lang Painless

• Fix Bad Casts In Painless #23282 (issue: #23238)
• Don’t allow casting from void to def in painless #22969 (issue: #22908)
• Fix def invoked qualified method refs #22918

Plugins

• Add check for null pluginName in remove command #22930 (issue: #22922)

Query DSL

• Fix parsing for max_determinized_states #22749 (issue: #22722)

REST

• [API] change wait_for_completion default according to docs #23672
• HTTP transport stashes the ThreadContext instead of the RestController #23456
• Ensure we try to autodetect content type for handlers that support plain text #23452
• Fix date format in warning headers #23418 (issue: #23275)
• Correct warning header to be compliant #23275 (issue: #22986)
• Fix search scroll request with a plain text body #23183 (issue: #22691)
• Handle bad HTTP requests #23153 (issue: #23034)
• Properly encode location header #23133 (issues: #21057, #23115)

Reindex API

• Reindex: do not log when can’t clear old scroll #22942 (issue: #22937)
• Fix reindex-from-remote from <2.0 #22931 (issue: #22893)

Scripting

• Remove unnecessary Groovy deprecation logging #23410 (issue: #23401)
• Script: Fix value of ctx._now to be current epoch time in milliseconds #23175 (issue: #23169)

Search

• Honor max concurrent searches in multi-search #23538 (issue: #23527)
• Avoid stack overflow in multi-search #23527 (issue: #23523)
• Replace blocking calls in ExpandCollapseSearchResponseListener by asynchronous requests #23053 (issue: #23048)
• Ensure fixed serialization order of InnerHitBuilder #22820 (issue: #22808)

Settings

• Fix merge scheduler config settings #23391
• Settings: Fix keystore cli prompting for yes/no to handle console returning null #23320
• Expose search.highlight.term_vector_multi_value as a node level setting #22999
• NPE when no setting name passed to elasticsearch-keystore #22609

Similarities

• Fix similarity upgrade when "default" similarity is overridden #23163

Stats

• Avoid overflow when computing total FS stats #23641
• Handle long overflow when adding paths' totals #23293 (issue: #23093)
• Fix control group pattern #23219 (issue: #23218)
• Fix total disk bytes returning negative value #23093

Task Manager

• Fix hanging cancelling task with no children #22796
• Fix broken TaskInfo.toString() #22698 (issue: #22387)

Regressions

Core

• Source filtering: only accept array items if the previous include pattern matches #22593 (issue: #22557)

Upgrades

Internal

• Upgrade to Lucene 6.4.1. #22978

memo-release-notes-5.3.1.md

5.3.1 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.3.

Breaking changes

Settings

• Remove support for default settings #24093 (issues: #23981, #24052, #24074)

Enhancements

Core

• Detect remnants of path.data/default.path.data bug #24099 (issues: #23981, #24052, #24074, #24093)

Network

• Adjust default Netty receive predictor size to 64k #23542 (issue: #23185)

REST

• Cluster stats should not render empty http/transport types #23735

Bug fixes

Aggregations

• The filter and significant_terms aggregations should parse the filter as a filter, not a query. #23797
• Restore support for the include/pattern syntax. #23140 (issue: #22933)

Bulk

• Reject empty IDs #24118 (issue: #24116)

Core

• Improve performance of extracting warning value #24114 (issue: #24018)
• Restrict build info loading to ES jar, not any jar #24049 (issue: #21955)

Discovery

• ZenDiscovery - only validate min_master_nodes values if local node is master #23915 (issue: #23695)

Index APIs

• Honor update request timeout #23825

Inner Hits

• Replace NestedChildrenQuery with ParentChildrenBlockJoinQuery #24039 (issue: #24016)

Internal

• Add infrastructure to mark contexts as system contexts #23830

Java REST Client

• Make buffer limit configurable in HeapBufferedConsumerFactory #23970 (issue: #23958)

Mapping

• Preserve response headers when creating an index #23950 (issue: #23947)
• Fix NPE with scaled floats stats when field is not indexed #23528 (issue: #23487)

Network

• Fix possible hang in local transport when nodes get concurrently disconnected #23962 (issue: #23942)

Plugin Repository Azure

• Azure blob store’s readBlob() method first checks if the blob exists #23483 (issue: #23480)

Reindex API

• Fix reindex with a remote source on a version before 2.0.0 #23805

Search

• Disable graph analysis at query time for shingle and cjk filters producing tokens of different size #23920 (issue: #23918)
• Fix cross-cluster remote node gateway attributes #23863
• Use a fixed seed for computing term hashCode in TermsSliceQuery #23795

Settings

• Do not set path.data in environment if not set #24132 (issue: #24099)
• Correct handling of default and array settings #24074 (issues: #23981, #24052)

Stats

• Handle existence of cgroup version 2 hierarchy #23493 (issue: #23486)

Upgrades

Internal

• upgrade to Lucene 6.4.2 #23933

memo-release-notes-5.3.2.md

5.3.2 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.3.

Bug fixes

Aggregations

• Align behavior HDR percentiles iterator with percentile() method #24206

Core

• Check for default.path.data included in path.data #24285 (issue: #24283)

Plugin Ingest Attachment

• Add Fix for java.lang.NoClassDefFoundError: com/graphbuilder/curve/Point fatal error in 5.3 [ISSUE] #24273

Plugins

• Fix delete of plugin directory on remove plugin #24266 (issue: #24252)
• Use a marker file when removing a plugin #24252 (issue: #24231)

Search Templates

• No longer add illegal content type option to stored search templates #24251 (issue: #24227)

Regressions

Bulk

• Fix _bulk response when it can’t create an index #24048 (issues: #22488, #24028)

memo-release-notes-5.3.3.md

5.3.3 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.3.

Enhancements

Network

• Disable the Netty recycler in the client #24793 (issues: #22452, #24721)

Bug fixes

Index APIs

• Fix legacy GeoPointField decoding in FieldStats #24534 (issue: #24275)

Packaging

• Handle parentheses in batch file path #24731 (issue: #24712)

Search

• Fix RangeFieldMapper rangeQuery to properly handle relations #24808 (issue: #24744)

memo-release-notes-5.4.0.md

5.4.0 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.4.

Breaking changes

Settings

• Remove support for default settings #24093 (issues: #23981, #24052, #24074)

Breaking Java changes

Aggregations

• Move getProperty method out of MultiBucketsAggregation.Bucket interface #23988
• Remove getProperty method from Aggregations interface and impl #23972
• Move getProperty method out of Aggregation interface #23949

Java API

• Fold InternalSearchHits and friends into their interfaces #23042

Deprecations

Aggregations

• Deprecate Stats#getCountAsString #24292

Java API

• Add BulkProcessor methods with XContentType parameter #23078 (issue: #22691)

Network

• Deprecate Netty 3 #23459 (issue: #23411)

Packaging

• Add deprecation warnings for $ES_USER and $ES_GROUP #24190 (issue: #23989)

Plugin Delete By Query

• Deprecate delete_by_query requests without an explicit query #23635

Plugin Repository Azure

• Deprecate global repositories.azure settings #22856 (issue: #22800)

Plugin Repository S3

• Deprecate repositories.s3 settings #23278 (issues: #22800, #23276)

REST

• Deprecate ldjson support and document ndjson for bulk/msearch #23051 (issues: #23025, #23049)

Stats

• Deprecate _field_stats endpoint #23914

New features

Analysis

• Adds pattern keyword marker filter support #23600 (issue: #4877)
• Expose WordDelimiterGraphTokenFilter #23327 (issue: #23104)

Index APIs

• Add FieldCapabilities (_field_caps) API #23007 (issue: #22438)

Search

• Introduce incremental reduction of TopDocs #23946

Similarities

• Adds boolean similarity to Elasticsearch #23637 (issue: #6731)

Enhancements

Aggregations

• Add BucketMetricValue interface #24188
• Move aggs CommonFields and TYPED_KEYS_DELIMITER from InternalAggregation to Aggregation #23987
• Use ParseField for aggs CommonFields rather than String #23717
• Share XContent rendering code in terms aggs #23680
• Add unit tests for ParentToChildAggregator #23305 (issue: #22278)
• First step towards incremental reduction of query responses #23253

Allocation

• Trigger replica recovery restarts by master when primary relocation completes #23926 (issue: #23904)
• Makes the same_shard host dynamically updatable #23397 (issue: #22992)

Analysis

• Support Keyword type in Analyze API #23161

Cluster

• Prevent nodes from joining if newer indices exist in the cluster #23843

Core

• Detect remnants of path.data/default.path.data bug #24099 (issues: #23981, #24052, #24074, #24093)
• Await termination after shutting down executors #23889
• Add early-access check #23743 (issue: #23668)
• Adapter action future should restore interrupts #23618 (issue: #23617)
• Disable bootstrap checks for single-node discovery #23598 (issues: #23585, #23595)
• Enable explicitly enforcing bootstrap checks #23585 (issue: #21864)
• Add equals/hashcode method to ReplicationResponse #23215

Dates

• Improve error handling for epoch format parser with time zone (#22621) #23689

Discovery

• Introduce single-node discovery #23595
• UnicastZenPing shouldn’t ping the address of the local node #23567
• MasterFaultDetection can start after the initial cluster state has been processed #23037 (issue: #22828)

Highlighting

• Add support for fragment_length in the unified highlighter #23431
• Add BreakIteratorBoundaryScanner support #23248

Index APIs

• Wildcard cluster names for cross cluster search #23985 (issue: #23893)

Ingest

• Lazy load the geoip databases #23337

Internal

• Add a dedicated TransportRemoteInfoAction for consistency #24040 (issue: #23969)
• Simplify sorted top docs merging in SearchPhaseController #23881
• Synchronized CollapseTopFieldDocs with lucenes relatives #23854
• Cleanup SearchPhaseController interface #23844
• Do not create String instances in Strings methods accepting StringBuilder #22907

Java API

• Added types options to DeleteByQueryRequest #23265 (issue: #21984)

Java High Level REST Client

• Convert suggestion response parsing to use NamedXContentRegistry #23355
• UpdateRequest implements ToXContent #23289
• Add javadoc for DocWriteResponse.Builders #23267
• Expose WriteRequest.RefreshPolicy string representation #23106
• Use typed_keys parameter to prefix suggester names by type in search responses #23080 (issue: #22965)
• Add parsing methods to BulkItemResponse #22859

Logging

• Warn on not enough masters during election #20063 (issue: #8362)

Mapping

• Improve error message for ipv6 on legacy ip fields #23136 (issue: #23126)

Nested Docs

• Avoid adding unnecessary nested filters when ranges are used. #23427

Network

• Adjust default Netty receive predictor size to 64k #23542 (issue: #23185)
• Keep the pipeline handler queue small initially #23335
• Set network receive predictor size to 32kb #23284 (issue: #23185)
• TransportService.connectToNode should validate remote node ID #22828 (issue: #22194)

Packaging

• Introduce Java version check #23194 (issue: #21102)
• Cleanup some things after removal of joda-time hack #18959

Percolator

• Allowing range queries with now ranges inside percolator queries #23921 (issue: #23859)
• Add term extraction support for MultiPhraseQuery #23176

Plugin Discovery EC2

• Settings: Migrate ec2 discovery sensitive settings to elasticsearch keystore #23961 (issue: #22475)

Plugin Lang Painless

• Allow painless to load stored fields #24290
• Start on custom whitelists for Painless #23563
• Fix Painless’s implementation of interfaces returning primitives #23298 (issue: #22983)
• Allow painless to implement more interfaces #22983

Plugin Repository Azure

• Add Backoff policy to azure repository #23387 (issue: #22728)

Plugin Repository S3

• Removes the retry mechanism from the S3 blob store #23952 (issue: #22845)
• S3 Repository: Eagerly load static settings #23910

Plugins

• Modify permissions dialog for plugins #23742
• Plugins: Add plugin cli specific exit codes #23599 (issue: #15295)
• Plugins: Output better error message when existing plugin is incompatible #23562 (issue: #20691)

Query DSL

• Make it possible to validate a query on all shards instead of a single random shard #23697 (issue: #18254)

REST

• Validate top-level keys when parsing mget requests #23746 (issue: #23720)
• Cluster stats should not render empty http/transport types #23735
• Add parameter to prefix aggs name with type in search responses #22965

Search

• Set shard count limit to unlimited #24012
• Streamline shard index availability in all SearchPhaseResults #23788
• Search took time should use a relative clock #23662
• Prevent negative from parameter in SearchSourceBuilder #23358 (issue: #23324)
• Remove unnecessary result sorting in SearchPhaseController #23321
• Expose batched_reduce_size via _search #23288 (issue: #23253)
• Adding fromXContent to Suggest and Suggestion class #23226 (issue: #23202)
• Adding fromXContent to Suggestion.Entry and subclasses #23202
• Add CollapseSearchPhase as a successor for the FetchSearchPhase #23165
• Integrate IndexOrDocValuesQuery. #23119
• Detach SearchPhases from AbstractSearchAsyncAction #23118
• Fix GraphQuery expectation after Lucene upgrade to 6.5 #23117 (issue: #23102)
• Nested queries should avoid adding unnecessary filters when possible. #23079 (issue: #20797)
• Add xcontent parsing to completion suggestion option #23071
• Add xcontent parsing to suggestion options #23018
• Separate reduce (aggs, suggest and profile) from merging fetched hits #23017

Settings

• Add secure file setting to keystore #24001
• Add a setting which specifies a list of setting #23883
• Add a property to mark setting as final #23872
• Remove obsolete index setting index.version.minimum_compatible. #23593
• Provide a method to retrieve a closeable char[] from a SecureString #23389
• Update indices settings api to support CBOR and SMILE format #23309 (issues: #23242, #23245)
• Improve setting deprecation message #23156 (issue: #22849)

Snapshot/Restore

• Change snapshot status error to use generic SnapshotException #24355 (issue: #24225)

Stats

• Add cross-cluster search remote cluster info API #23969 (issue: #23925)

Task Manager

• Allow task to be unregistered by ClusterStateApplier #23931
• Limit IndexRequest toString() length #22832

Bug fixes

Aggregations

• Align behavior HDR percentiles iterator with percentile() method #24206
• The filter and significant_terms aggregations should parse the filter as a filter, not a query. #23797
• Completion suggestion should also consider text if prefix/regex is missing #23451 (issue: #23340)
• Fixes the per term error in the terms aggregation #23399
• Fixes terms error count for multiple reduce phases #23291 (issue: #23286)
• Restore support for the include/pattern syntax. #23140 (issue: #22933)

Bulk

• Reject empty IDs #24118 (issue: #24116)

CRUD

• Fix backport executing ops as single item bulk #23083 (issues: #21964, #23069)

Cluster

• Don’t set local node on cluster state used for node join validation #23311 (issues: #21830, #3, #4, #6, #9)
• Allow a cluster state applier to create an observer and wait for a better state #23132 (issue: #21817)
• Cluster allocation explain to never return empty response body #23054

Core

• Check for default.path.data included in path.data #24285 (issue: #24283)
• Improve performance of extracting warning value #24114 (issue: #24018)
• Reject duplicate settings on the command line #24053
• Restrict build info loading to ES jar, not any jar #24049 (issue: #21955)

Discovery

• ZenDiscovery - only validate min_master_nodes values if local node is master #23915 (issue: #23695)

Index APIs

• Fixes restore of a shrunken index when initial recovery node is gone #24322 (issue: #24257)
• Honor update request timeout #23825

Ingest

• Improve missing ingest processor error #23379 (issue: #23392)
• update _ingest.timestamp to use new ZonedDateTime #23174 (issue: #23168)

Inner Hits

• Replace NestedChildrenQuery with ParentChildrenBlockJoinQuery #24016 (issue: #24009)
• Changed DisMaxQueryBuilder to extract inner hits from leaf queries #23512 (issue: #23482)

Internal

• Add infrastructure to mark contexts as system contexts #23830
• Always restore the ThreadContext for operations delayed due to a block #23349

Java High Level REST Client

• Correctly parse BulkItemResponse.Failure’s status #23432

Java REST Client

• Make buffer limit configurable in HeapBufferedConsumerFactory #23970 (issue: #23958)
• RestClient asynchronous execution should not throw exceptions #23307

Mapping

• Preserve response headers when creating an index #23950 (issue: #23947)
• Improves disabled fielddata error message #23841 (issue: #22768)
• Switch include_in_all in multifield to warning #23656 (issues: #21971, #23654)
• Fix MapperService StackOverflowError #23605 (issue: #23604)
• Fix NPE with scaled floats stats when field is not indexed #23528 (issue: #23487)

Network

• Fix possible hang in local transport when nodes get concurrently disconnected #23962 (issue: #23942)
• Respect promises on pipelined responses #23317 (issues: #23310, #23322)
• Ensure that releasing listener is called #23310

Packaging

• Fall back to non-atomic move when removing plugins #23548 (issue: #35)

Percolator

• Fix memory leak when percolator uses bitset or field data cache #24115 (issue: #24108)

Plugin Ingest Attachment

• Remove support for Visio and potm files #23214 (issues: #22077, #22079, #22963)

Plugin Lang Painless

• Fix painless’s regex lexer and error messages #23634
• Replace Painless’s Cast with casting strategies #23369
• Fix Bad Casts In Painless #23282 (issue: #23238)

Plugin Repository Azure

• Azure blob store’s readBlob() method first checks if the blob exists #23483 (issue: #23480)

Plugin Repository S3

• Handle BlobPath’s trailing separator case. Add test cases to BlobPathTests.java #23091

Plugins

• Fix delete of plugin directory on remove plugin #24266 (issue: #24252)
• Use a marker file when removing a plugin #24252 (issue: #24231)

Query DSL

• FuzzyQueryBuilder should error when parsing array of values #23762 (issue: #23759)

REST

• [API] change wait_for_completion default according to docs #23672
• Deprecate request_cache for clear-cache #23638 (issue: #22748)
• HTTP transport stashes the ThreadContext instead of the RestController #23456
• Ensure we try to autodetect content type for handlers that support plain text #23452
• Fix date format in warning headers #23418 (issue: #23275)
• Align REST specs for HEAD requests #23313 (issue: #21125)
• Correct warning header to be compliant #23275 (issue: #22986)
• Fix get HEAD requests #23186 (issue: #21125)
• Fix search scroll request with a plain text body #23183 (issue: #22691)
• Handle bad HTTP requests #23153 (issue: #23034)
• Fix get source HEAD requests #23151 (issue: #21125)
• Properly encode location header #23133 (issues: #21057, #23115)
• Fix template HEAD requests #23130 (issue: #21125)
• Fix index HEAD requests #23112 (issue: #21125)
• Fix alias HEAD requests #23094 (issue: #21125)

Recovery

• Provide target allocation id as part of start recovery request #24333 (issue: #24167)

Reindex API

• Fix throttled reindex_from_remote #23953 (issues: #23828, #23945)
• Fix reindex with a remote source on a version before 2.0.0 #23805
• Make reindex wait for cleanup before responding #23677 (issue: #23653)

Scripting

• Remove unnecessary Groovy deprecation logging #23410 (issue: #23401)
• Convert script/template objects to json format internally #23308 (issue: #23245)
• Script: Fix value of ctx._now to be current epoch time in milliseconds #23175 (issue: #23169)

Search

• Cross Cluster Search: propagate original indices per cluster #24328
• Query string default field #24214
• Speed up parsing of large terms queries. #24210
• IndicesQueryCache should delegate the scorerSupplier method. #24209
• Fork LRUQueryCache from Lucene to work around LUCENE-7749 #24207
• Disable graph analysis at query time for shingle and cjk filters producing tokens of different size #23920 (issue: #23918)
• Fix cross-cluster remote node gateway attributes #23863
• Use a fixed seed for computing term hashCode in TermsSliceQuery #23795
• Honor max concurrent searches in multi-search #23538 (issue: #23527)
• Avoid stack overflow in multi-search #23527 (issue: #23523)
• Fix query_string_query to transform "foo:*" in an exists query on the field name #23433 (issue: #23356)
• Factor out filling of TopDocs in SearchPhaseController #23380 (issues: #19356, #23357)
• Replace blocking calls in ExpandCollapseSearchResponseListener by asynchronous requests #23053 (issue: #23048)

Search Templates

• No longer add illegal content type option to stored search templates #24251 (issue: #24227)

Settings

• Do not set path.data in environment if not set #24132 (issue: #24099)
• Correct handling of default and array settings #24074 (issues: #23981, #24052)
• Fix merge scheduler config settings #23391
• Settings: Fix keystore cli prompting for yes/no to handle console returning null #23320

Similarities

• Fix similarity upgrade when "default" similarity is overridden #23163

Snapshot/Restore

• Fixes maintaining the shards a snapshot is waiting on #24289
• Fixes snapshot status on failed snapshots #23833 (issue: #23716)
• Fixes snapshot deletion handling on in-progress snapshot failure #23703 (issue: #23663)
• Prioritize listing index-N blobs over index.latest in reading snapshots #23333

Stats

• Avoid overflow when computing total FS stats #23641
• Handle existence of cgroup version 2 hierarchy #23493 (issue: #23486)
• Handle long overflow when adding paths' totals #23293 (issue: #23093)
• Fix control group pattern #23219 (issue: #23218)
• Fix total disk bytes returning negative value #23093

Regressions

Bulk

• Fix _bulk response when it can’t create an index #24048 (issues: #22488, #24028)

Upgrades

Aggregations

• Upgrade HDRHistogram to 2.1.9 #23254 (issue: #23239)

Core

• Upgrade to Lucene 6.5.0 #23750
• Upgrade from JNA 4.2.2 to JNA 4.4.0 #23636
• Upgrade to lucene-6.5.0-snapshot-d00c5ca #23385
• Upgrade to lucene-6.5.0-snapshot-f919485. #23087

Logging

• Upgrade to Log4j 2.8.2 #23995

Network

• Upgrade to Netty 4.1.9 #23540 (issues: #23172, #6308, #6374)
• Upgrade to Netty 4.1.8 #23055

Plugin Repository Azure

• Update to Azure Storage 5.0.0 #23517 (issue: #23448)

memo-release-notes-5.4.1.md

5.4.1 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.4.

Enhancements

Aggregations

• Allow scripted metric agg to access _score #24295

Core

• Improve bootstrap checks error messages #24548

Internal

• Log JVM arguments on startup #24451

Network

• Disable the Netty recycler in the client #24793 (issues: #22452, #24721)
• Remove Netty logging hack #24653 (issues: #24469, #5624, #6568, #6696)
• Set available processors for Netty #24420 (issue: #6224)

Snapshot/Restore

• Ensure every repository has an incompatible-snapshots blob #24403 (issue: #22267)

Stats

• Show JVM arguments #24450

Bug fixes

Aggregations

• Fix ArrayIndexOutOfBoundsException when no ranges are specified in the query #23241 (issue: #22881)

Allocation

• Discard stale node responses from async shard fetching #24434 (issue: #24007)

Core

• Fix cache expire after access #24546

Index APIs

• Fix legacy GeoPointField decoding in FieldStats #24534 (issue: #24275)
• Validates updated settings on closed indices #24487 (issue: #23787)

Internal

• Remove _UNRELEASED from Version constants for released versions #24764
• Avoid race when shutting down controller processes #24579
• Fix NPE if field caps request has a field that exists not in all indices #24504

More Like This

• Pass over _routing value with more_like_this items to be retrieved #24679 (issue: #23699)

Network

• Notify onConnectionClosed rather than onNodeDisconnect to prune transport handlers #24639 (issues: #24557, #24575, #24632)
• SniffNodesSampler should close connection after handling responses #24632 (issues: #22828, #24557, #24575)

Packaging

• Handle parentheses in batch file path #24731 (issue: #24712)

Percolator

• Fix range queries with date range based on current time in percolator queries. #24666 (issue: #23921)

Plugin Repository HDFS

• Fixing permission errors for KERBEROS security mode for HDFS Repository #23439 (issue: #22156)

Plugins

• X-Pack plugin download fails on Windows desktop [ISSUE] #24570
• Fix plugin installation permissions #24527 (issue: #24480)

Reindex API

• Reindex: don’t duplicate _source parameter #24629 (issue: #24628)
• Add qa module that tests reindex-from-remote against pre-5.0 versions of Elasticsearch #24561 (issues: #23828, #24520)

Scroll

• Fix single shard scroll within a cluster with nodes in version >= 5.3 and <= 5.3 #24512

Search

• Fix ExpandSearchPhase when response contains no hits #24688 (issue: #24672)
• Include all aliases including non-filtering in _search_shards response #24489

Settings

• Allow resetting settings that use an IP validator #24713 (issue: #24709)

Snapshot/Restore

• Removes completed snapshot from cluster state on master change #24605 (issue: #24452)
• Fix inefficient (worst case exponential) loading of snapshot repository #24510 (issue: #24509)

Upgrades

Core

• Upgrade to Lucene 6.5.1 #24399

Network

• Upgrade to Netty 4.1.11.Final #24652
• Upgrade Netty to 4.1.10.Final #24414

memo-release-notes-5.4.2.md

5.4.2 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.4.

Enhancements

Core

• Explicitly reject duplicate data paths #25178

Packaging

• Set number of processes in systemd unit file #24970 (issue: #20874)

Bug fixes

Aggregations

• Terms aggregation should remap global ordinal buckets when a sub-aggregator is used to sort the terms #24941 (issue: #24788)

Analysis

• PatternAnalyzer should lowercase wildcard queries when lowercase is true. #24967

Circuit Breakers

• Checks the circuit breaker before allocating bytes for a new big array #25010 (issue: #24790)

Geo

• Fix GeoPoint FieldStats ternary logic bug #25211 (issue: #24275)
• Fix GeoDistance Ordinal for BWC #25033 (issue: #24816)

Index APIs

• Rollover max docs should only count primaries #24977 (issue: #24217)

Nested Docs

• Fix the max score mode. #25216 (issue: #24647)

Network

• Ensure pending transport handlers are invoked for all channel failures #25150

Packaging

• When stopping via systemd only kill the JVM, not its control group #25195

Suggesters

• Fix context suggester to read values from keyword type field #24200 (issue: #24129)

memo-release-notes-5.4.3.md

5.4.3 Release Notes

Enhancements

Snapshot/Restore

• Improves snapshot logging and snapshot deletion error handling #25264

Bug fixes

Snapshot/Restore

• Remove redundant and broken MD5 checksum from repository-s3 #25270 (issue: #25269)

memo-release-notes-5.5.0.md

5.5.0 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.5.

Breaking changes

Plugins

• Remove hidden file leniency from spawner #24079 (issue: #23980)

Settings

• Remove support for default settings #24093 (issues: #23981, #24052, #24074)

Breaking Java changes

Aggregations

• Remove the unused SignificantTerms.compareTerm() method #24714
• Make SignificantTerms.Bucket an interface rather than an abstract class #24670 (issue: #24492)
• Make Terms.Bucket an interface rather than an abstract class #24492

Plugin Delete By Query

• Move DeleteByQuery and Reindex requests into core #24578

Deprecations

Index APIs

• Deprecated use of + in index expressions #24585 (issue: #24515)

Packaging

• Add deprecation warnings for $ES_USER and $ES_GROUP #24190 (issue: #23989)

Plugin Repository S3

• S3 Repository: Deprecate remaining repositories.s3.* settings #24144 (issue: #24143)

REST

• Add deprecation logging for comma-separated feature parsing #25040 (issue: #24723)

Scripting

• Scripting: Deprecate native scripts #24692 (issue: #19966)
• Scripting: Deprecate index lookup #24691 (issue: #19359)
• Deprecate Fine Grain Settings for Scripts #24573 (issue: #24532)
• Scripting: Deprecate file script settings #24555 (issue: #21798)
• Scripting: Deprecate file scripts #24552 (issue: #21798)

Settings

• Settings: Update settings deprecation from yml to yaml #24663 (issue: #19391)
• Deprecate settings in .yml and .json #24059 (issue: #19391)

Tribe Node

• Deprecate tribe service #24598 (issue: #24581)

New features

Mapping

• Add new ip_range field type #24433

Plugin Analysis ICU

• Add ICUCollationFieldMapper #24126

Enhancements

Aggregations

• Add superset size to Significant Term REST response #24865
• Add document count to Matrix Stats aggregation response #24776
• Share XContent rendering code in significant terms aggregations #24677
• Add count to rest output of geo_centroid #24387 (issue: #24366)
• Allow scripted metric agg to access _score #24295

Aliases

• Handle multiple aliases in _cat/aliases api #23698 (issue: #23661)

Circuit Breakers

• Closing a ReleasableBytesStreamOutput closes the underlying BigArray #23941

Cluster

• Add a cluster block that allows to delete indices that are read-only #24678
• Extract a common base class to allow services to listen to remote cluster config updates #24367

Core

• Explicitly reject duplicate data paths #25178
• Use SPI in High Level Rest Client to load XContent parsers #25097
• Improve bootstrap checks error messages #24548
• Regex upgrades #24316 (issue: #24226)
• Detect remnants of path.data/default.path.data bug #24099 (issues: #23981, #24052, #24074, #24093)

Engine

• Add refresh stats tracking for realtime get #25052 (issue: #24806)
• Engine: store maxUnsafeAutoIdTimestamp in commit #24149

Exceptions

• IllegalStateException: Only duplicated jar instead of classpath #24953

Index APIs

• Open and close index to honour allow_no_indices option #24222 (issue: #24031)

Ingest

• add exclude_keys option to KeyValueProcessor #24876 (issue: #23856)

Inner Hits

• Reuse inner hit query weight #24571 (issue: #23917)

Internal

• "shard started" should show index and shard ID #25157
• Add the ability to set eager_global_ordinals in the new parent-join field #25019
• Add assertions enabled helper #24834
• Add cross cluster support to _field_caps #24463 (issue: #24334)
• Log JVM arguments on startup #24451
• Preserve cluster alias throughout search execution to lookup nodes by cluster and ID #24438
• Move RemoteClusterService into TransportService #24424
• Enum related performance additions. #24274 (issue: #24226)

Java API

• Always Accumulate Transport Exceptions #25017 (issue: #23099)

Java High Level REST Client

• Add doc_count to ParsedMatrixStats #24952 (issue: #24776)
• Add fromXContent method to ClearScrollResponse #24909
• ClearScrollRequest to implement ToXContentObject #24907
• SearchScrollRequest to implement ToXContentObject #24906 (issue: #3889)
• Backport aggs parsers for high level REST Client #24844 (issues: #22278, #22668, #23826, #23965, #23973, #23974, #24085, #24155, #24160, #24162, #24182, #24183, #24208, #24213, #24239, #24284, #24291, #24312, #24330, #24365, #24371, #24442, #24521, #24524, #24559, #24564, #24583, #24589, #24648, #24667, #24675, #24682, #24700, #24706, #24717, #24720, #24738, #24746, #24789, #24791, #24794, #24822, #24824)
• Add utility method to parse named XContent objects with typed prefix #24240 (issue: #22965)

Logging

• Warn on not enough masters during election #20063 (issue: #8362)

Mapping

• Do not index _type when there is at most one type. #24363
• Only allow one type on 6.0 indices #24317 (issue: #15613)
• token_count type : add an option to count tokens (fix #23227) #24175 (issue: #23227)

Network

• Disable the Netty recycler in the client #24793 (issues: #22452, #24721)
• Remove Netty logging hack #24653 (issues: #24469, #5624, #6568, #6696)
• Set available processors for Netty #24420 (issue: #6224)

Packaging

• Set number of processes in systemd unit file #24970 (issue: #20874)

Plugin Lang Painless

• Optimize instance creation in LambdaBootstrap #24618
• Allow painless to load stored fields #24290

Plugin Repository GCS

• GCS Repository: Add secure storage of credentials #24697

Plugins

• Add purge option to remove plugin CLI #24981
• Move ReindexAction class to core #24684 (issue: #24578)
• Plugins: Add support for platform specific plugins #24265

REST

• Make ObjectParser support string to boolean conversion #24668 (issue: #21802)
• Allow passing single scrollID in clear scroll API body #24242 (issue: #24233)

Scripting

• Add Ability to Specify No Types/Contexts Allowed For Scripts #24782
• Add New Security Script Settings #24637 (issue: #24532)

Search

• Eliminate array access in tight loops when profiling is enabled. #24959
• Support Multiple Inner Hits on a Field Collapse Request #24517
• Expand cross cluster search indices for search requests to the concrete index or to it’s aliases #24502
• Remove leniency when merging fetched hits in a search response phase #24158

Search Templates

• Add max concurrent searches to multi template search #24255 (issues: #20912, #21907)

Settings

• Emit settings deprecation logging at most once #25457

Snapshot/Restore

• Improves snapshot logging and snapshot deletion error handling #25264
• Enhances get snapshots API to allow retrieving repository index only #24477 (issue: #24288)
• Ensure every repository has an incompatible-snapshots blob #24403 (issue: #22267)
• Change snapshot status error to use generic SnapshotException #24355 (issue: #24225)

Stats

• Show JVM arguments #24450

Suggesters

• Provide informative error message in case of unknown suggestion context. #24241
• Allow different data types for category in Context suggester #23491 (issue: #22358)

Upgrade API

• Allow plugins to upgrade templates and index metadata on startup #24379

Bug fixes

Aggregations

• Terms aggregation should remap global ordinal buckets when a sub-aggregator is used to sort the terms #24941 (issue: #24788)
• Correctly set doc_count when MovAvg "predicts" values on existing buckets #24892 (issue: #24327)
• InternalPercentilesBucket should not rely on ordered percents array #24336 (issue: #24331)
• Align behavior HDR percentiles iterator with percentile() method #24206
• DateHistogram: Fix extended_bounds with offset #23789 (issue: #23776)
• Fix ArrayIndexOutOfBoundsException when no ranges are specified in the query #23241 (issue: #22881)

Aliases

• GET aliases should 404 if aliases are missing #25043 (issue: #24644)

Allocation

• Discard stale node responses from async shard fetching #24434 (issue: #24007)

Analysis

• PatternAnalyzer should lowercase wildcard queries when lowercase is true. #24967

Bulk

• Reject empty IDs #24118 (issue: #24116)

Cache

• fix bug of weight computation #24856

Circuit Breakers

• Checks the circuit breaker before allocating bytes for a new big array #25010 (issue: #24790)

Core

• Fix cache expire after access #24546
• Check for default.path.data included in path.data #24285 (issue: #24283)

Geo

• Fix GeoPoint FieldStats ternary logic bug #25211 (issue: #24275)
• Fix GeoDistance Ordinal for BWC #25033 (issue: #24816)

Highlighting

• Higlighters: Fix MultiPhrasePrefixQuery rewriting #25103 (issue: #25088)

Index APIs

• Rollover max docs should only count primaries #24977 (issue: #24217)
• Fix legacy GeoPointField decoding in FieldStats #24534 (issue: #24275)
• Validates updated settings on closed indices #24487 (issue: #23787)
• Fixes restore of a shrunken index when initial recovery node is gone #24322 (issue: #24257)

Ingest

• fix grok’s pattern parsing to validate pattern names in expression #25063 (issue: #22831)
• Fix floating-point error when DateProcessor parses UNIX #24947
• add option for _ingest.timestamp to use new ZonedDateTime (5.x backport) #24030 (issues: #23168, #23174)

Inner Hits

• Fix Source filtering in new field collapsing feature #24068 (issue: #24063)

Internal

• Fix race condition in RemoteClusterConnection node supplier #25432
• Initialise empty lists in BaseTaskResponse constructor #25290
• Extract a common base class for scroll executions #24979 (issue: #16555)
• Ensure remote cluster is connected before fetching _field_caps #24845 (issue: #24763)
• Obey lock order if working with store to get metadata snapshots #24787 (issue: #24481)
• Remove _UNRELEASED from Version constants for released versions #24764
• Fix _field_caps serialization in order to support cross cluster search #24722
• Avoid race when shutting down controller processes #24579
• Fix NPE if field caps request has a field that exists not in all indices #24504

More Like This

• Pass over _routing value with more_like_this items to be retrieved #24679 (issue: #23699)

Nested Docs

• In case of a single type the _id field should be added to the nested document instead of _uid field #25149

Network

• Ensure pending transport handlers are invoked for all channel failures #25150
• Notify onConnectionClosed rather than onNodeDisconnect to prune transport handlers #24639 (issues: #24557, #24575, #24632)
• SniffNodesSampler should close connection after handling responses #24632 (issues: #22828, #24557, #24575)

Packaging

• When stopping via systemd only kill the JVM, not its control group #25195
• Handle parentheses in batch file path #24731 (issue: #24712)

Percolator

• Fix range queries with date range based on current time in percolator queries. #24666 (issue: #23921)
• For legacy indices rewrite percolator query upon percolation time #24617 (issue: #24485)
• Fix memory leak when percolator uses bitset or field data cache #24115 (issue: #24108)

Plugin Lang Painless

• painless: Fix method references to ctor with the new LambdaBootstrap and cleanup code #24406
• Fix Painless Lambdas for Java 9 #24070 (issue: #23473)

Plugins

• X-Pack plugin download fails on Windows desktop #24570
• Fix plugin installation permissions #24527 (issue: #24480)
• Fix delete of plugin directory on remove plugin #24266 (issue: #24252)
• Use a marker file when removing a plugin #24252 (issue: #24231)
• Remove hidden file leniency from plugin service #23982 (issue: #12465)

Query DSL

• Thread falls into infinite loop when processing Indices query #24736 (issue: #24735)

REST

• Fix handling of exceptions thrown on HEAD requests #25172 (issue: #21125)
• Fixed NPEs caused by requests without content. #23497 (issue: #24701)
• Fix get mappings HEAD requests #23192 (issue: #21125)

Recovery

• Provide target allocation id as part of start recovery request #24333 (issue: #24167)

Reindex API

• Reindex: don’t duplicate _source parameter #24629 (issue: #24628)
• Add qa module that tests reindex-from-remote against pre-5.0 versions of Elasticsearch #24561 (issues: #23828, #24520)

Scroll

• Fix single shard scroll within a cluster with nodes in version >= 5.3 and <= 5.3 #24512

Search

• Adds check for negative search request size #25397 (issue: #22530)
• Do not search locally if remote index pattern resolves to no indices #25436 (issue: #25426)
• Make sure range queries are correctly profiled. #25108
• Compute the took time of the query after the expand phase of field collapsing #24902 (issue: #24900)
• Fix RangeFieldMapper rangeQuery to properly handle relations #24808 (issue: #24744)
• Fix ExpandSearchPhase when response contains no hits #24688 (issue: #24672)
• Include all aliases including non-filtering in _search_shards response #24489
• Cross Cluster Search: propagate original indices per cluster #24328

Search Templates

• No longer add illegal content type option to stored search templates #24251 (issue: #24227)

Settings

• Keystore CLI should use the AddFileKeyStoreCommand for files #25298
• Settings: Fix setting groups to include secure settings #25076 (issue: #25069)
• Allow resetting settings that use an IP validator #24713 (issue: #24709)
• Do not set path.data in environment if not set #24132 (issue: #24099)

Snapshot/Restore

• Output all empty snapshot info fields if in verbose mode #25455 (issue: #24477)
• Remove redundant and broken MD5 checksum from repository-s3 #25270 (issue: #25269)
• Consolidates the logic for cleaning up snapshots on master election #24894 (issue: #24605)
• Removes completed snapshot from cluster state on master change #24605 (issue: #24452)
• Fix inefficient (worst case exponential) loading of snapshot repository #24510 (issue: #24509)
• Fixes maintaining the shards a snapshot is waiting on #24289

Stats

• _nodes/stats should not fail due to concurrent AlreadyClosedException #25016 (issue: #23099)
• Avoid double decrement on current query counter #24922 (issues: #22996, #24872)

Suggesters

• Fix context suggester to read values from keyword type field #24200 (issue: #24129)

Task Manager

• Task Management: Make TaskInfo parsing forwards compatible #24073 (issue: #23250)

Regressions

Highlighting

• Fix Fast Vector Highlighter NPE on match phrase prefix #25116 (issue: #25088)

Upgrades

Core

• Upgrade to Lucene 6.5.1 #24399

Network

• Upgrade to Netty 4.1.11.Final #24652
• Upgrade Netty to 4.1.10.Final #24414

memo-release-notes-5.5.1.md

5.5.1 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.5.

Bug fixes

Aliases

• mget with an alias shouldn’t ignore alias routing #25697 (issue: #25696)

Engine

• Engine - Do not store operations that are not index into lucene in the translog (5.x only) #25592

Internal

• Fix BytesReferenceStreamInput#skip with offset #25634

Packaging

• Stop disabling explicit GC #25759
• Avoid failing install if system-sysctl is masked #25657 (issue: #24234)
• Get short path name for native controllers #25344

Query DSL

• WrapperQueryBuilder should also rewrite the parsed query #25480

memo-release-notes-5.5.2.md

5.5.2 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.5.

Bug fixes

Aggregations

• Fixes array out of bounds for value count agg #26038 (issue: #17379)

Core

• Release operation permit on thread-pool rejection #25930 (issue: #25863)

Inner Hits

• Fix inner hits to work with queries wrapped in an indices query #26138 (issue: #26133)
• When fetching nested inner hits only access stored fields when needed #25864 (issue: #6)

Logging

• Declare XContent deprecation logger as static #25881 (issue: #25879)

Query DSL

• Parse "*" in query_string_query as MatchAllDocsQuery #25872 (issues: #25556, #25726)

memo-release-notes-5.5.3.md

5.5.3 Release Notes

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.5.

Deprecations

Mapping

• Add deprecation logging when _all is enabled #26228 (issue: #22144)

Bug fixes

Packaging

• setgid on /etc/elasticearch on package install #26412 (issue: #26410)

memo-release-notes-5.6.0.md

5.6.0 Release Notes

Also see Breaking changes in 5.6.

Breaking Java changes

Java High Level REST Client

• Allow RestHighLevelClient to use plugins #25024

Java REST Client

• Rename client artifacts #25693 (issue: #20248)

Deprecations

Highlighting

• Postings highlighter deprecation #25073

Indexed Scripts/Templates

• Scripting: Deprecate stored search template apis #25437 (issue: #24596)

Mapping

• Add deprecation logging when _all is enabled #26228 (issue: #22144)

Query DSL

• Deprecate omitting terms lookup index name in 5.6 #25755 (issue: #25750)

Scripting

• Scripting: Deprecate scripts.max_compilation_per_minute setting #26402
• Scripting: Change keys for inline/stored scripts to source/id #25127

Tribe Node

• Deprecate tribe service #24598 (issue: #24581)

New features

Java High Level REST Client

• Backport high level REST client #25086 (issues: #22371, #22680, #23040, #23187, #23266, #23312, #23328, #23350, #23351, #23373, #24796, #24938, #25038)

Parent/Child

• Typeless parent child backport #25104
• Move parent_id query to the parent-join module #25072 (issue: #20257)
• Introduce ParentJoinFieldMapper, a field mapper that creates parent/child relation within documents of the same index #24978 (issue: #20257)

Upgrade API

• TemplateUpgraders should be called during rolling restart #25263 (issues: #24379, #24680)

Enhancements

Allocation

• Adjust status on bad allocation explain requests #25503 (issue: #25458)
• Promote replica on the highest version node #25277 (issue: #10708)

Analysis

• [analysis-icu] Allow setting unicodeSetFilter #20814 (issue: #20820)

CAT API

• expand /_cat/nodes to return information about hard drive #21775 (issue: #21679)

Cluster

• Validate a joining node’s version with version of existing cluster nodes #25808

Core

• Add max file size bootstrap check #25974
• Explicitly reject duplicate data paths #25178

Highlighting

• Picks offset source for the unified highlighter directly from the es mapping #25747 (issue: #25699)

Index APIs

• Create index request should return the index name #25139 (issue: #23044)

Ingest

• Add Ingest-Processor specific Rest Endpoints & Add Grok endpoint #25059 (issue: #24725)
• Port support for commercial GeoIP2 databases from Logstash. #24889
• Allow removing multiple fields in ingest processor #24750 (issue: #24622)
• Add target_field parameter to ingest processors #24133 (issues: #23228, #23682)

Internal

• Use holder pattern for lazy deprecation loggers #26218 (issue: #26210)
• "shard started" should show index and shard ID #25157
• Break out clear scroll logic from TransportClearScrollAction #25125 (issue: #25094)
• Add helper methods to TransportActionProxy to identify proxy actions and requests #25124
• Add remote cluster infrastructure to fetch discovery nodes. #25123 (issue: #25094)
• Disallow multiple parent-join fields per mapping #25002

Java High Level REST Client

• [DOCS] restructure java clients docs pages #25517
• Use SPI in High Level Rest Client to load XContent parsers #25098 (issues: #25024, #25097)

Java REST Client

• Shade external dependencies in the rest client jar #25780 (issue: #25208)
• RestClient uses system properties and system default SSLContext #25757 (issue: #23231)

Logging

• Use LRU set to reduce repeat deprecation messages #25474 (issue: #25457)

Mapping

• Optimize terms queries on ip addresses to use a PointInSetQuery whenever possible. #25669 (issue: #25667)

Network

• Simplify connection closing and cleanups in TcpTransport #25250

Packaging

• Remove memlock suggestion from systemd service #25979
• Set address space limit in systemd service file #25975

Parent/Child

• Remove ParentJoinFieldSubFetchPhase #25550 (issue: #25363)
• Support parent id being specified as number in the _source #25547

Query DSL

• Make slop optional when parsing span_near query #25677 (issue: #25642)

Search

• Reduce the overhead of timeouts and low-level search cancellation. #25776
• Reduce profiling overhead. #25772 (issue: #24799)
• Backport can_match endpoint to 5.6 to allow 6.0 to use the optimization in mixed version #25704 (issue: #25658)
• Add a shard filter search phase to pre-filter shards based on query rewriting #25658
• Ensure we rewrite common queries to match_none if possible #25650
• Limit the number of concurrent shard requests per search request #25632
• Add cluster name validation to RemoteClusterConnection #25568
• Leverage scorerSupplier when applicable. #25109
• Add Cross Cluster Search support for scroll searches #25094

Sequence IDs

• 5.x - Move primary term from ReplicationRequest to ConcreteShardRequest #25824 (issue: #25822)
• Forward compatibility for primary context handoff on 6.x #25545 (issue: #25468)

Settings

• Emit settings deprecation logging at most once #25457
• IndexMetaData: Introduce internal format index setting #25292

Snapshot/Restore

• Improves snapshot logging and snapshot deletion error handling #25264

Stats

• Update IndexShard#refreshMetric via a ReferenceManager.RefreshListener #25083 (issues: #24806, #25052)

Bug fixes

Aggregations

• Fixes array out of bounds for value count agg #26038 (issue: #17379)

Aliases

• mget with an alias shouldn’t ignore alias routing #25697 (issue: #25696)

Allocation

• Allow wildcards for shard IP filtering #26187 (issues: #22591, #26184)

Analysis

• Pre-configured shingle filter should disable graph analysis #25853 (issue: #25555)

CRUD

• Serialize and expose timeout of acknowledged requests in REST layer (ES 5.6) #26213 (issue: #26189)

Cluster

• Register setting cluster.indices.tombstones.size #26193 (issue: #26191)

Core

• Release operation permit on thread-pool rejection #25930 (issue: #25863)
• Node should start up despite of a lingering .es_temp_file #21210 (issue: #21007)

Discovery

• MasterNodeChangePredicate should use the node instance to detect master change #25877 (issue: #25471)

Engine

• Engine - Do not store operations that are not index into lucene in the translog (5.x only) #25592

Geo

• Fix GeoPoint FieldStats ternary logic bug #25211 (issue: #24275)

Highlighting

• Fix nested query highlighting #26305 (issue: #26230)
• FastVectorHighlighter should not cache the field query globally #25197 (issue: #25171)

Index APIs

• Shrink API should ignore templates #25380 (issue: #25035)
• Rollover max docs should only count primaries #24977 (issue: #24217)

Ingest

• Sort Processor does not have proper behavior with targetField #25237 (issue: #24133)

Inner Hits

• Fix inner hits to work with queries wrapped in an indices query #26138 (issue: #26133)
• When fetching nested inner hits only access stored fields when needed #25864 (issue: #6)

Internal

• Fix BytesReferenceStreamInput#skip with offset #25634
• Fix race condition in RemoteClusterConnection node supplier #25432
• Initialise empty lists in BaseTaskResponse constructor #25290

Java High Level REST Client

• Register ip_range aggregation with the high level client #26383
• add top hits as a parsed aggregation to the rest high level client #26370

Logging

• Lazy initialize deprecation logger in parser #26210 (issue: #25879)
• Allow not configure logging without config #26209 (issues: #20575, #24076)
• Declare XContent deprecation logger as static #25881 (issue: #25879)

Mapping

• ICUCollationKeywordFieldMapper use SortedSetDocValuesField #26267
• Fixed bug that mapper_parsing_exception is thrown for numeric field with ignore_malformed=true when inserting "NaN" #25967 (issue: #25289)
• Coerce decimal strings for whole number types by truncating the decimal part #25835 (issue: #25819)
• Fix parsing of ip range queries. #25768 (issue: #25636)
• Remove fielddata:false from mappings of non-analyzed strings. #25542
• Disable date field mapping changing #25285 (issue: #25271)
• token_count datatype should handle null value #25046 (issue: #24928)
• keep _parent field while updating child type mapping #24407 (issue: #23381)

Nested Docs

• In case of a single type the _id field should be added to the nested document instead of _uid field #25149

Network

• Release pipelined http responses on close #26226
• Ensure pending transport handlers are invoked for all channel failures #25150

Packaging

• setgid on /etc/elasticearch on package install #26412 (issue: #26410)
• Stop disabling explicit GC #25759
• Avoid failing install if system-sysctl is masked #25657 (issue: #24234)
• Get short path name for native controllers #25344
• When stopping via systemd only kill the JVM, not its control group #25195

Parent/Child

• The default _parent field should not try to load global ordinals #25851 (issue: #25849)

Plugin Lang Painless

• Painless: allow doubles to be casted to longs. #25936

Plugin Repository HDFS

• Upgrading HDFS Repository Plugin to use HDFS 2.8.1 Client #25497 (issue: #25450)

Query DSL

• Parse "*" in query_string_query as MatchAllDocsQuery #25872 (issues: #25556, #25726)
• SpanNearQueryBuilder should return the inner clause when a single clause is provided #25856 (issue: #25630)
• WrapperQueryBuilder should also rewrite the parsed query #25480

REST

• Fix handling of exceptions thrown on HEAD requests #25172 (issue: #21125)

Search

• Fix term(s) query for range field #25918
• Caching a MinDocQuery can lead to wrong results. #25909
• First increment shard stats before notifying and potentially sending response #25818
• Remove assertion about deviation when casting to a float. #25806 (issue: #25330)
• Prevent skipping shards if a suggest builder is present #25739 (issue: #25658)
• Ensure remote cluster alias is preserved in inner hits aggs #25627 (issue: #25606)
• Do not search locally if remote index pattern resolves to no indices #25436 (issue: #25426)

Settings

• Fix settings serialization to not serialize secure settings or not take the total size into account #25323
• Keystore CLI should use the AddFileKeyStoreCommand for files #25298

Snapshot/Restore

• Snapshot/Restore: fix NPE while handling null failure reasons #26127 (issues: #25878, #25941)
• Snapshot/Restore: Ensure that shard failure reasons are correctly stored in CS #25941 (issue: #25878)
• Output all empty snapshot info fields if in verbose mode #25455 (issue: #24477)

Stats

• _nodes/stats should not fail due to concurrent AlreadyClosedException #25016 (issue: #23099)

Suggesters

• Context suggester should filter doc values field #25858 (issue: #25404)

Upgrades

Network

• Upgrade to Netty 4.1.13.Final #25581 (issues: #24729, #6866)

Upgrade API

• Improve stability and logging of TemplateUpgradeServiceIT tests #25386 (issue: #25382)