derms’s gists

derms / builld.gradle

Created March 7, 2019 10:59

Count Lines in CSV's before MLCP

	task importSampleData(type: com.marklogic.gradle.task.MlcpTask) {
	def directory = "./sample-dir/" // can get this from gradle properties

	//parse csv files
	doFirst {
	println "\nSTART: Reading files"
	def totalLineCount = 0
	fileTree(directory).each{ file ->
	def filename = file.getName()
	def lineCount = java.nio.file.Files.lines(file.toPath()).count() -1;

derms / test-if-server-configured.sh

Created March 13, 2019 15:40

Test if ML server is already configred

	echo "Testing if already configured"
	ADMIN_STATUS_CODE=`curl -I http://localhost:8001/admin/v1/init 2>/dev/null \| head -n 1 \| cut -d$' ' -f2`
	if [ "$ADMIN_STATUS_CODE" == "401" ]; then
	echo "Server is already configured."
	exit 0
	fi

derms / kill-sticky.js

Created March 21, 2019 15:05 — forked from alisdair/kill-sticky.js

Kill sticky headers.

	(function () {
	var i, elements = document.querySelectorAll('body *');

	for (i = 0; i < elements.length; i++) {
	if (getComputedStyle(elements[i]).position === 'fixed') {
	elements[i].parentNode.removeChild(elements[i]);
	}
	}
	})();

derms / rep

Last active March 25, 2019 10:54

replica


	* Documents
	• App-Services
	• Modules
	• Schemas
	• Triggers
	• Security


	curl --anyauth --user admin:XXXX -H "Content-Type: application/json" -X POST -d '{"forest-name": "Documents-replica-01","host": "myserver-02"}' http://localhost:8002/manage/v2/forests

derms / merge.fio

Last active March 25, 2019 14:41

perf

derms / mlcp.options

Created March 26, 2019 12:47

mlcp

derms / build.gradle

Last active April 19, 2019 11:12

MarkLogic - Multiple Delete Collections tasks

	class CustomDeleteCollectionsTask extends com.marklogic.gradle.task.datamovement.DataMovementTask {
	String[] collections

	@org.gradle.api.tasks.TaskAction
	void deleteCollections() {
	project.ext.collections = (collections != null && collections.length > 0) ? collections.join(",") : null
	runQueryBatcherJob(new com.marklogic.client.ext.datamovement.job.DeleteCollectionsJob())
	}
	}

derms / sample.sh

Created May 22, 2019 15:04

Find files with certain extension(s) and find the size of those files

 find . -name "*.pdf" -o -name "*.jpg" -o -name "*.png" -o -name "*.tif" -o -name "*.PNG" | sed 's| |\\ |g' | xargs ls -sk  | sed 's/ .\/.*\//,/g'

derms / clear-caches.xqy

Created July 18, 2019 12:32

marklogic clear caches

	xquery version "1.0-ml";
	(xdmp:list-cache-clear(),
	xdmp:expanded-tree-cache-clear(),
	xdmp:compressed-tree-cache-clear(),
	xdmp:triple-cache-clear(),
	xdmp:triple-value-cache-clear(),
	xdmp:cache-status(xdmp:host()))

derms / 1_import_onnx_model.sjs

Last active February 27, 2020 15:25

Example of using ONNX Machine Learning Models in MarkLogic 10 (10.0-4 and above). BiDAF model: https://github.com/onnx/models/tree/master/text/machine_comprehension/bidirectional_attention_flow.

	'use strict';
	// Simple way to download the model and insert it into the database.
	// Normally would use a tool like mlcp to do this. This only needs to be done once

	declareUpdate()
	//downloads the BiDAF model
	let model = fn.subsequence(
	xdmp.httpGet("https://onnxzoo.blob.core.windows.net/models/opset_9/bidaf/bidaf.onnx")
	, 2, 1)
	//inserts the BiDAF model into the database at the URI /onnx/model/bidaf.onnx