Skip to content

Instantly share code, notes, and snippets.

@tharun208

tharun208/cockroach-after.bash

Created March 1, 2021 10:19
Show Gist options
  • Save tharun208/1b56a47ec7fd3a4f2857efc85897105a to your computer and use it in GitHub Desktop.
Save tharun208/1b56a47ec7fd3a4f2857efc85897105a to your computer and use it in GitHub Desktop.
Download ZIP
cockroach-after
Raw
cockroach-after.bash
Start a CockroachDB node, which will export data from one or more
storage devices, specified via --store flags.
Specify the --join flag to point to another node or nodes that are
part of the same cluster. The other nodes do not need to be started
yet, and if the address of the other nodes to be added are not yet
known it is legal for the first node to join itself.
To initialize the cluster, use 'cockroach init'.
Usage:
cockroach start [flags]
Examples:
cockroach start --insecure --store=attrs=ssd,path=/mnt/ssd1 --join=host:port,[host:port]
Flags:
--advertise-addr <addr/host>[:<port>] The address/hostname and port to advertise to other CockroachDB nodes
for intra-cluster communication. It must resolve and be routable from
other nodes in the cluster.
If left unspecified, it defaults to the setting of --listen-addr. If
the flag is provided but either the address part or the port part is
left unspecified, that particular part defaults to the same part in
--listen-addr.
An IPv6 address can also be specified with the notation [...], for
example [::1]:26257 or [fe80::f6f2:::]:26257.
The port number should be the same as in --listen-addr unless port
forwarding is set up on an intermediate firewall/router.
(default :)
--advertise-sql-addr <addr/host>[:<port>] The SQL address/hostname and port to advertise to CLI admin utilities
and via SQL introspection for the purpose of SQL address discovery. It
must resolve and be routable from clients.
If left unspecified, it defaults to the setting of --sql-addr. If
the flag is provided but either the address part or the port part is
left unspecified, that particular part defaults to the same part in
--sql-addr.
An IPv6 address can also be specified with the notation [...], for
example [::1]:26257 or [fe80::f6f2:::]:26257.
The port number should be the same as in --sql-addr unless port
forwarding is set up on an intermediate firewall/router.
(default :)
--attrs string An ordered, colon-separated list of node attributes. Attributes are
arbitrary strings specifying machine capabilities. Machine capabilities
might include specialized hardware or number of cores (e.g. "gpu",
"x16c"). For example:
--attrs=x16c:gpu
--background Start the server in the background. This is similar to appending "&"
to the command line, but when the server is started with --background,
control is not returned to the shell until the server is ready to
accept requests.
--cache bytes Total size in bytes for caches, shared evenly if there are multiple
storage devices. Size suffixes are supported (e.g. 1GB and 1GiB). If
left unspecified, defaults to 128MiB. A percentage of physical memory
can also be specified (e.g. .25).
(default 128 MiB)
--cert-principal-map strings A comma separated list of <cert-principal>:<db-principal> mappings.
This allows mapping the principal in a cert to a DB principal such
as "node" or "root" or any SQL user. This is intended for use in
situations where the certificate management system places restrictions
on the Subject.CommonName or SubjectAlternateName fields in the
certificate (e.g. disallowing a CommonName such as "node" or "root").
If multiple mappings are provided for the same <cert-principal>,
the last one specified in the list takes precedence. A principal
not specified in the map is passed through as-is via the identity
function. A cert is allowed to authenticate a DB principal if the
DB principal name is contained in the mapped CommonName or DNS-type
SubjectAlternateName fields.
--certs-dir string Path to the directory containing SSL certificates and keys.
(default "${HOME}/.cockroach-certs")
--clock-device string Override HLC to use PTP hardware clock user space API when querying
for current time. The value corresponds to the clock device to be used.
This is currently only tested and supported on Linux.
--clock-device=/dev/ptp0
--cluster-name <identifier> Sets a name to verify the identity of a remote node or cluster. The
value must match between this node and the remote node(s) specified via
--join.
This can be used as an additional verification when either the node
or cluster, or both, have not yet been initialized and do not yet know
their cluster ID.
To introduce a cluster name into an already-initialized cluster, pair
this flag with --disable-cluster-name-verification.
--disable-cluster-name-verification Tell the server to ignore cluster name mismatches. This is meant for
use when opting an existing cluster into starting to use cluster name
verification, or when changing the cluster name.
The cluster should be restarted once with --cluster-name and
--disable-cluster-name-verification combined, and once all nodes have
been updated to know the new cluster name, the cluster can be restarted
again with this flag removed.
--enterprise-encryption StoreEncryptionSpec Specify encryption options for one of the stores on a node. If multiple
stores exist, the flag must be specified for each store.
A valid enterprise license is required to use this functionality.
Key files must be of size 32 bytes + AES key size, such as:
AES-128: 48 bytes
AES-192: 56 bytes
AES-256: 64 bytes
Valid fields:
* path (required): must match the path of one of the stores
* key (required): path to the current key file, or "plain"
* old-key (required): path to the previous key file, or "plain"
* rotation-period : amount of time after which data keys should be rotated
example:
--enterprise-encryption=path=cockroach-data,key=/keys/aes-128.key,old-key=plain
--experimental-dns-srv When enabled, the node will first attempt to fetch SRV records from
DNS for every name specified with --join. If a valid SRV record is
found, that information is used instead of regular DNS A/AAAA lookups.
This feature is experimental and may be removed or modified in a later
version.
--external-io-dir string The local file path under which remotely-initiated operations that can
specify node-local I/O paths, such as BACKUP, RESTORE or IMPORT, can
access files. Following symlinks _is_ allowed, meaning that other paths
can be added by symlinking to them from within this path.
Note: operations in a distributed cluster can run across many nodes,
so reading or writing to any given node's local file system in a
distributed cluster is not usually useful unless that filesystem is
actually backed by something like NFS.
If left empty, defaults to the "extern" subdirectory of the first store
directory.
The value "disabled" will disable all local file I/O.
--external-io-disable-http Disable use of HTTP when accessing external data.
--external-io-disable-implicit-credentials Disable use of implicit credentials when accessing external data.
Instead, require the user to always specify access keys.
--external-io-disabled Disable use of "external" IO, such as to S3, GCS, or the file system
(nodelocal), or anything other than userfile.
-h, --help help for start
--http-addr <addr/host>[:<port>] The hostname or IP address to bind to for HTTP requests. If
left unspecified, the address part defaults to the setting of
--listen-addr. The port number defaults to 8080. An IPv6 address can
also be specified with the notation [...], for example [::1]:8080 or
[fe80::f6f2:::]:8080.
(default :8080)
--insecure Start a node with all security controls disabled. There is no
encryption, no authentication and internal security checks are also
disabled. This makes any client able to take over the entire cluster.
This flag is only intended for non-production testing.
Beware that using this flag on a public network without --listen-addr
is likely to cause the entire host server to become compromised.
To simply accept non-TLS connections for SQL clients while keeping the
cluster secure, consider using --accept-sql-without-tls instead. Also
see: https://go.crdb.dev/issue-v/53404/v21.1
-j, --join string The addresses for connecting a node to a cluster.
When starting a multi-node cluster for the first time, set this flag to
the addresses of 3-5 of the initial nodes. Then run the cockroach init
command against one of the nodes to complete cluster startup.
When starting a singe-node cluster, leave this flag out. This will
cause the node to initialize a new single-node cluster without needing
to run the cockroach init command.
When adding a node to an existing cluster, set this flag to 3-5 of
the nodes already in the cluster; it's easiest to use the same list of
addresses that was used to start the initial nodes.
This flag can be specified separately for each address:
--join=localhost:1234 --join=localhost:2345
Or can be specified as a comma separated list in single flag, or both
forms can be used together, for example:
--join=localhost:1234,localhost:2345 --join=localhost:3456
--listen-addr <addr/host>[:<port>] The address/hostname and port to listen on for intra-cluster
communication, for example --listen-addr=myhost:26257 or
--listen-addr=:26257 (listen on all interfaces). Unless --sql-addr
is also specified, this address is also used to accept SQL client
connections.
If the address part is left unspecified, it defaults to the "all
interfaces" address (0.0.0.0 IPv4 / [::] IPv6). If the port part is
left unspecified, it defaults to 26257.
An IPv6 address can also be specified with the notation [...], for
example [::1]:26257 or [fe80::f6f2:::]:26257.
If --advertise-addr is left unspecified, the node will also announce
this address for use by other nodes. It is strongly recommended to use
--advertise-addr in cloud and container deployments or any setup where
NAT is present between cluster nodes.
(default :26257)
--listening-url-file string After the CockroachDB node has started up successfully, it will write
its connection URL to the specified file.
--locality Locality An ordered, comma-separated list of key-value pairs that describe the
topography of the machine. Topography might include country, datacenter
or rack designations. Data is automatically replicated to maximize
diversities of each tier. The order of tiers is used to determine
the priority of the diversity, so the more inclusive localities like
country should come before less inclusive localities like datacenter.
The tiers and order must be the same on all nodes. Including more tiers
is better than including fewer. For example:
--locality=country=us,region=us-west,datacenter=us-west-1b,rack=12
--locality=country=ca,region=ca-east,datacenter=ca-east-2,rack=4
--locality=planet=earth,province=manitoba,colo=secondary,power=3
--locality-advertise-addr localityList List of ports to advertise to other CockroachDB nodes for intra-cluster
communication for some locality. This should be specified as a commma
separated list of locality@address. Addresses can also include ports.
For example:
"[email protected],[email protected]"
"[email protected]:26257,[email protected]:26258"
--max-disk-temp-storage bytes Maximum storage capacity available to store temporary disk-based data
for SQL queries that exceed the memory budget (e.g. join, sorts, etc
are sometimes able to spill intermediate results to disk). Accepts
numbers interpreted as bytes, size suffixes (e.g. 32GB and 32GiB) or
a percentage of disk size (e.g. 10%). If left unspecified, defaults to
32GiB.
The location of the temporary files is within the first store dir (see
--store). If expressed as a percentage, --max-disk-temp-storage is
interpreted relative to the size of the storage device on which the
first store is placed. The temp space usage is never counted towards
any store usage (although it does share the device with the first
store) so, when configuring this, make sure that the size of this temp
storage plus the size of the first store don't exceed the capacity of
the storage device.
If the first store is an in-memory one (i.e. type=mem), then this
temporary "disk" data is also kept in-memory. A percentage value
is interpreted as a percentage of the available internal memory. If
not specified, the default shifts to 100MiB when the first store is
in-memory.
--max-offset MaxOffset Maximum allowed clock offset for the cluster. If observed clock offsets
exceed this limit, servers will crash to minimize the likelihood of
reading inconsistent data. Increasing this value will increase the time
to recovery of failures as well as the frequency of uncertainty-based
read restarts.
Note that this value must be the same on all nodes in the cluster.
In order to change it, all nodes in the cluster must be stopped
simultaneously and restarted with the new value.
(default 500ms)
--max-sql-memory bytes Maximum memory capacity available to store temporary data for SQL
clients, including prepared queries and intermediate data rows during
query execution. Accepts numbers interpreted as bytes, size suffixes
(e.g. 1GB and 1GiB) or a percentage of physical memory (e.g. .25). If
left unspecified, defaults to 25% of physical memory.
(default 4.0 GiB)
--pid-file string After the CockroachDB node has started up successfully, it will write
its process ID to the specified file.
--socket-dir string Accept client connections using a Unix domain socket created in
the specified directory. Note: for compatibility with PostgreSQL
clients and drivers, the generated socket name has the form
"/path/to/.s.PGSQL.NNNN", where NNNN is the port number configured
via --listen-addr. PostgreSQL clients only take a port number and
directory as input and construct the socket name programmatically. To
use, for example:
psql -h /path/to -p NNNN ...
Environment variable: COCKROACH_SOCKET_DIR
--spatial-libs string The location where all libraries for spatial operations is located.
(default "/usr/local/lib/cockroach")
--sql-addr <addr/host>[:<port>] The hostname or IP address to bind to for SQL clients, for
example --sql-addr=myhost:26257 or --sql-addr=:26257 (listen on
all interfaces). If left unspecified, the address specified by
--listen-addr will be used for both RPC and SQL connections.
If specified but the address part is omitted, the address part defaults
to the address part of --listen-addr. If specified but the port number
is omitted, the port number defaults to 26257.
To actually use separate bindings, it is recommended to specify both
flags and use a different port number via --listen-addr, for example
--sql-addr=:26257 --listen-addr=:26258. Ensure that --join is set
accordingly on other nodes. It is also possible to use the same port
number but separate host addresses.
An IPv6 address can also be specified with the notation [...], for
example [::1]:26257 or [fe80::f6f2:::]:26257.
(default :)
--storage-engine string Storage engine to use for all stores on this cockroach node. The only
option is pebble. Deprecated; only present for backward compatibility.
(default "pebble")
-s, --store StoreSpec The file path to a storage device. This flag must be specified
separately for each storage device, for example:
--store=/mnt/ssd01 --store=/mnt/ssd02 --store=/mnt/hda1
For each store, the "attrs" and "size" fields can be used to specify
device attributes and a maximum store size (see below). When one or
both of these fields are set, the "path" field label must be used for
the path to the storage device, for example:
--store=path=/mnt/ssd01,attrs=ssd,size=20GiB
In most cases, node-level attributes are preferable to store-level
attributes. However, the "attrs" field can be used to match
capabilities for storage of individual databases or tables. For
example, an OLTP database would probably want to allocate space for
its tables only on solid state devices, whereas append-only time
series might prefer cheaper spinning drives. Typical attributes include
whether the store is flash (ssd), spinny disk (hdd), or in-memory
(mem), as well as speeds and other specs. Attributes can be arbitrary
strings separated by colons, for example:
--store=path=/mnt/hda1,attrs=hdd:7200rpm
The store size in the "size" field is not a guaranteed maximum but is
used when calculating free space for rebalancing purposes. The size can
be specified either in a bytes-based unit or as a percentage of hard
drive space, for example:
--store=path=/mnt/ssd01,size=10000000000 -> 10000000000 bytes
--store=path=/mnt/ssd01,size=20GB -> 20000000000 bytes
--store=path=/mnt/ssd01,size=20GiB -> 21474836480 bytes
--store=path=/mnt/ssd01,size=0.02TiB -> 21474836480 bytes
--store=path=/mnt/ssd01,size=20% -> 20% of available space
--store=path=/mnt/ssd01,size=0.2 -> 20% of available space
--store=path=/mnt/ssd01,size=.2 -> 20% of available space
For an in-memory store, the "type" and "size" fields are required,
and the "path" field is forbidden. The "type" field must be set to
"mem", and the "size" field must be set to the true maximum bytes or
percentage of available memory that the store may consume, for example:
--store=type=mem,size=20GiB
--store=type=mem,size=90%
Commas are forbidden in all values, since they are used to separate
fields. Also, if you use equal signs in the file path to a store, you
must use the "path" field label.
(default --store=path=/Users/tharun/go/src/github.com/cockroachdb/cockroach/cockroach-data)
--temp-dir string The parent directory path where a temporary subdirectory will be
created to be used for temporary files. This path must exist or the
node will not start. The temporary subdirectory is used primarily
as working memory for distributed computations and CSV importing.
For example, the following will generate an arbitrary, temporary
subdirectory "/mnt/ssd01/temp/cockroach-temp<NUMBER>":
--temp-dir=/mnt/ssd01/temp
If this flag is unspecified, the temporary subdirectory will be located
under the root of the first store.
--unencrypted-localhost-http When specified, restricts HTTP connections to localhost-only and
disables TLS for the HTTP interface. The hostname part of --http-addr,
if specified, is then ignored. This flag is intended for use to
facilitate local testing without requiring certificate setups in web
browsers.
Global Flags:
--log <string> Logging configuration. See the documentation for details.
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging (significantly hurts performance)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment