Created
November 9, 2022 14:09
-
-
Save Inkimar/1e186a98726d212846161dee5bcafd80 to your computer and use it in GitHub Desktop.
2021-11-9, SNIC - biocache -> dr632
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 1. load has been done | |
| 2. biocache> process -dr dr632 |
Failed to connect to Zookeeper
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | java.net.UnknownHostException: zookeeper3: Name or service not known
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at java.net.InetAddress.getAllByName(InetAddress.java:1193)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at java.net.InetAddress.getAllByName(InetAddress.java:1127)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at java.net.InetAddress.getByName(InetAddress.java:1077)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer.recreateSocketAddresses(QuorumPeer.java:181)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:591)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:610)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:838)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:958)
solrcloud_zookeeper1.1.bkgihw11jnv1@uat-solr-1 | 2022-11-09 14:28:11,236 [myid:1] - INFO [QuorumPeer[myid=1]/0.0.0.0:2181:FastLeaderElection@847] - Notification time out: 12800
Try to restart with this command:
- docker service update --force solrcloud_solr1
- docker service update --force solrcloud_solr4
- docker service update --force solrcloud_solr2
- docker service update --force solrcloud_solr3
Error when it comes to mounting the disk
docker service update --force solrcloud_zookeeper1
solrcloud_zookeeper1
overall progress: 0 out of 1 tasks
1/1: failed to mount local volume: mount /docker/var/volumes/solrcloud/data_zoo…
service update paused: update paused due to failure or early termination of task jt8o9waf5o7hprxhmzmclmwsk
2022-11-09
ingimar@uat-manager-1:~$ docker service ps solrcloud_zookeeper1
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
71cr8nk8eqp3 solrcloud_zookeeper1.1 zookeeper:3.4.14 uat-solr-1 Running Running 26 minutes ago *:3888->3888/tcp,*:3888->3888/tcp,*:2888->2888/tcp,*:2888->2888/tcp,*:2181->2181/tcp,*:2181->2181/tcp
x0bvmny1q5ts \_ solrcloud_zookeeper1.1 zookeeper:3.4.14 uat-manager-3 Shutdown Rejected 26 minutes ago "failed to mount local volume:…"
zbo6mq2m4n6y \_ solrcloud_zookeeper1.1 zookeeper:3.4.14 uat-manager-3 Shutdown Rejected 26 minutes ago "failed to mount local volume:…"
qo17l309mlwb \_ solrcloud_zookeeper1.1 zookeeper:3.4.14 uat-solr-4 Shutdown Rejected 26 minutes ago "failed to mount local volume:…"
ql8rh45jsg5t \_ solrcloud_zookeeper1.1 zookeeper:3.4.14 uat-solr-4 Shutdown Rejected 26 minutes ago "failed to mount local volume:…"
Jag kan se att disken finns på uat-solr1
Size Used Avail Use% Mounted on
udev 16G 0 16G 0% /dev
tmpfs 3.2G 1.5M 3.2G 1% /run
/dev/vda1 20G 12G 7.3G 63% /
tmpfs 16G 0 16G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 16G 0 16G 0% /sys/fs/cgroup
/dev/loop1 56M 56M 0 100% /snap/core18/2620
/dev/mapper/vg_docker_local-lv_var_lib_docker 79G 19G 56G 25% /docker_var_lib
/dev/loop0 56M 56M 0 100% /snap/core18/2566
/dev/loop3 64M 64M 0 100% /snap/core20/1634
/dev/loop4 71M 71M 0 100% /snap/lxd/19647
/dev/loop5 68M 68M 0 100% /snap/lxd/22753
/dev/loop6 48M 48M 0 100% /snap/snapd/17336
/dev/loop7 48M 48M 0 100% /snap/snapd/17029
/dev/mapper/vg_docker_local-lv_docker_local 200G 1.7G 199G 1% /docker_local
/dev/vda15 105M 5.2M 100M 5% /boot/efi
uat-manager-1:/docker 796G 491G 306G 62% /docker
/dev/loop8 64M 64M 0 100% /snap/core20/1695
tmpfs 3.2G 0 3.2G 0% /run/user/1700
Fungerar nu klockan 16:35 - 2021-11-09
biocache> index -dr dr632
There is an incremental row key file for this resource. Would you like to perform an incremental process (y/n) ?
y
7e1638f6c05d 2022-11-09 15:33:48,009 INFO : [IndexRecords] - Starting the reindex by row key....
7e1638f6c05d 2022-11-09 15:33:48,102 INFO : [SolrIndexDAO] - Initialising the solr server zookeeper1:2181,zookeeper2:2181 cloudserver:null solrServer:null
7e1638f6c05d 2022-11-09 15:33:49,477 INFO : [TaxonSpeciesListDAO] - Loading list information...
7e1638f6c05d 2022-11-09 15:33:49,958 INFO : [TaxonSpeciesListDAO] - Loading data from dr174
7e1638f6c05d 2022-11-09 15:33:49,959 INFO : [TaxonSpeciesListDAO] - Downloading list data from https://lists.biodiversitydata.se/speciesListItem/downloadList/dr174?fetch=%7BkvpValues%3Dselect%7D
7e1638f6c05d 2022-11-09 15:33:50,301 INFO : [TaxonSpeciesListDAO] - Loading data from dr204
7e1638f6c05d 2022-11-09 15:33:50,301 INFO : [TaxonSpeciesListDAO] - Downloading list data from https://lists.biodiversitydata.se/speciesListItem/downloadList/dr204?fetch=%7BkvpValues%3Dselect%7D
7e1638f6c05d 2022-11-09 15:34:16,035 INFO : [TaxonSpeciesListDAO] - Loading list information...completed.
7e1638f6c05d 2022-11-09 15:34:16,343 INFO : [SpeciesGroups] - Using default legacy species groups
7e1638f6c05d 2022-11-09 15:34:24,219 INFO : [IndexRecords] - 1000 >> Last key : 4840f76f-f82c-4f65-b276-ae432b79eaec, records per sec: 27.617443
7e1638f6c05d 2022-11-09 15:34:29,020 INFO : [IndexRecords] - 2000 >> Last key : 50cb9031-f2ab-4d49-9eeb-334ce2943953, records per sec: 208.33333
7e1638f6c05d 2022-11-09 15:34:33,188 INFO : [IndexRecords] - 3000 >> Last key : 696978cb-183a-4387-9357-3be6e8420098, records per sec: 239.92322
7e1638f6c05d 2022-11-09 15:34:37,362 INFO : [IndexRecords] - 4000 >> Last key : dcc58958-561b-404e-99a0-f4ec286abbdc, records per sec: 239.63576
7e1638f6c05d 2022-11-09 15:34:41,972 INFO : [IndexRecords] - 5000 >> Last key : 07214217-36cf-4da8-81c0-0be5ab070a1f, records per sec: 216.9668
7e1638f6c05d 2022-11-09 15:34:46,001 INFO : [IndexRecords] - 6000 >> Last key : 522f08a8-9ce9-47e7-b589-5c2c710dfa81, records per sec: 248.20056
7e1638f6c05d 2022-11-09 15:34:50,078 INFO : [IndexRecords] - 7000 >> Last key : 524e2ff9-b0f3-423a-b5d9-13b1fe7ba6fa, records per sec: 245.33855
7e1638f6c05d 2022-11-09 15:34:54,052 INFO : [IndexRecords] - 8000 >> Last key : 613d9f3c-6f84-4e3a-a3ca-5ec8d4c935a3, records per sec: 251.63564
7e1638f6c05d 2022-11-09 15:34:57,989 INFO : [IndexRecords] - 9000 >> Last key : a11d46be-0227-4711-b4f7-75a8f6462474, records per sec: 254.0005
7e1638f6c05d 2022-11-09 15:35:01,950 INFO : [IndexRecords] - 10000 >> Last key : cced1cdc-77ff-4c33-bf4b-1ef18d6b5f41, records per sec: 252.4615
7e1638f6c05d 2022-11-09 15:35:06,087 INFO : [IndexRecords] - 11000 >> Last key : f214a31d-8b8a-4c01-9a2f-98db86dc630c, records per sec: 241.7795
7e1638f6c05d 2022-11-09 15:35:07,607 INFO : [SolrIndexDAO] - Performing index commit....
7e1638f6c05d 2022-11-09 15:35:11,397 INFO : [SolrIndexDAO] - Performing index commit....done
7e1638f6c05d 2022-11-09 15:35:11,544 INFO : [SolrIndexDAO] - >>>> Document count of index: 16343
7e1638f6c05d 2022-11-09 15:35:11,545 INFO : [SolrIndexDAO] - Shutting down the indexing...
7e1638f6c05d 2022-11-09 15:35:11,663 INFO : [SolrIndexDAO] - Finalise finished.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Stop -> /opt/sbdi/bin/stop_docker_app solrcloud
Start -> /opt/sbdi/bin/start_docker_app solrcloud
docker service logs -f solrcloud_solr1