RyanSnodgrass

--- 
- 
  become: true
  become_user: root
  hosts: all
  tasks: 
    - 
      name: "Install openJDK 8"
      yum: 
        name: java-1.8.0-openjdk

RyanSnodgrass

sudo /usr/local/share/neo4j/bin/neo4j stop
sudo yum install libselinux-python
sudo yum java-1.8.0-openjdk
## select

RyanSnodgrass

Taking clues from 2.3.9 release notes and 2.3.8 deployment upgrade

following 3.0 docs

Run these commands from the '/usr/local/share' directory

To 2.3.9

1. shut down the database sudo /usr/local/share/neo4j/bin/neo4j stop or neop
2. wget dist.neo4j.org/neo4j-community-2.3.9-unix.tar.gz
3. tar -xzf neo4j-community-2.3.9-unix.tar.gz
4. mv neo4j-community-2.3.9 neo4j

RyanSnodgrass

Upgrading neo4j.rb to 8.x

Clear out old schema

The old way of doing schemas in neo4j.rb involved adding an index property on the model. This is no longer valid and you have to utilize the migration feature added in 8.0.

You can browse the schema on the localhost:747/browser window in your browser by running the :schema command

RyanSnodgrass

client = TinyTds::Client.new username: ENV["SQL_SERVER_USERNAME"],
                             password: ENV["SQL_SERVER_PASSWORD"],
                             port: ENV["SQL_SERVER_PORT"],
                             host: ENV["SQL_SERVER_HOST"],
                             database: ENV["SQL_SERVER_DB"]
query = "SELECT TOP(60000) [Institution of Higher Education Key]
     ,[Date Key]
     ,[Academic Term Key]
     ,[IPEDS Demographics Key]
     ,[IPEDS Fall Enrollment Attribute Key]

RyanSnodgrass

progress = 'Progress ['
1000.times do |i|
 
# i is number from 0-999
j = i + 1
 
  # add 1 percent every 10 times
  if j % 10 == 0
    progress << "="
    # move the cursor to the beginning of the line with \r

RyanSnodgrass

                       User       System     Total     Real
Suckerpunch-rake-local 31.710000   9.360000  41.070000 ( 45.723500)
Suckerpunch-rake-local 30.150000  12.850000  43.000000 ( 47.214349)
Suckerpunch-rake-local 31.840000  13.110000  44.950000 ( 49.994546)
Suckerpunch-rake-local 31.340000   9.300000  40.640000 ( 44.950706)
Suckerpunch-rake-local 29.520000   9.120000  38.640000 ( 42.868375)

sidekiq-rake-local     29.310000  12.860000  42.170000 ( 46.303059)
sidekiq-rake-local     29.810000   8.920000  38.730000 ( 42.672683)
sidekiq-rake-local     29.900000   9.040000  38.940000 ( 43.434940)

RyanSnodgrass

@db.extension(:sequel_3_dataset_methods)
report = MemoryProfiler.report do
  execute_as_user do
    File.open('dataset_exports/to_csv_method_mem.csv', 'w') do |f|
      f << @db[@query].to_csv.freeze
    end
  end
end
report.pretty_print

RyanSnodgrass

report = MemoryProfiler.report do
  execute_as_user do
    File.open('dataset_exports/to_csv_method_mem.csv', 'w') do |f|
      f << @db[query].to_csv
    end
  end
end
report.pretty_print

Total allocated: 196877053 bytes (2173575 objects)

RyanSnodgrass

report = MemoryProfiler.report do
  execute_as_user do
    CSV.open('dataset_exports/freeze_each_row_memory.csv', 'w') do |csv|
      csv << @db[query].first.keys
      @db[query].each do |row|
        csv << row.values.freeze
      end
    end
  end
end