Notes on OpenLDAP migration

05oct2016

Today's Research
================
OpenDJ Setup steps in Setup.py
------------------------------
* installObject.configure_opendj_install()
  - sets the load ldif command
  - sets the ldap_backend_type
  - writes the ldap password to file ldapPass
* installObject.setup_opendj()
  - add_ldap_schema
  - copy opendj-setup.properties to /opt/opendj
  - Run the setup command for opendj [/bin/su ldap -c setupCmd]
  - create the init script for the os to start the server upon boot
* installObject.configure_opendj()
  - setup some configs like
    > single-structural-objectclass-behaviour
    > password policy
    > log publisher
    > create-backend o=site
    > set Ldap connection handler to False
    > access control handler prop removed
    > reject-unauthenticated requests = true
    > Default password storage-schema = SHA 512 Salted
* index_opendj(backend) backends = "userRoot", "site"
  - set the proper index command
  - loads the mappings from the index.json
  - run the index command for each attribute given in the JSON
* import_ldif()
  - a set of files are stored as self.ldif_files
  - copy them to the ldapBaseFolder/ldif folder
  - ldifLoad these ldif files into the LDAP directory
  - import o_site.ldif
* delete the ldappassword file

06oct2016

Setting up the Symas build
===========================
1. The OpenLDAP is installed.
2. Copy the schemas to known locations /opt/gluu/gluu.schema, /opt/gluu/custom.schema, /opt/gluu/user.schema
3. Copy the slapd.conf and symas-openldap.conf
4. Importing the LDIF Files

Errors while importing
----------------------
root@openldap-testbed:~/community-edition-setup/output# /opt/symas/bin/slapadd -n 1 -l people.ldif 
slapadd: dn="inum=@!6DAE.FEBA.7F70.30F3!0001!6CAE.9969!0000!A8F2.DE1E.D7FB,ou=people,o=@!6DAE.FEBA.7F70.30F3!0001!6CAE.9969,o=gluu" 
(line=1): (65) invalid structural object class chain (gluuPerson/ox-6DAEFEBA7F7030F300016CAE9969)
_#################### 100.00% eta   none elapsed            none fast!         
Closing DB...

Refer this for resolving the error: http://www.openldap.org/faq/data/cache/883.html

-----------------------------------------------------------------------------------------------------------------------------
root@openldap-testbed:~/community-edition-setup/output# /opt/symas/bin/slapadd -n 1 -l ../static/cache-refresh/o_site.ldif 
slapadd: line 1: database #1 (o=gluu) not configured to hold "o=site"; no database configured for that naming context
_########              43.45% eta   none elapsed            none spd  94.1 k/s 
Closing DB...

-------------------------------------------------------------------------------------------------------------------------------
root@openldap-testbed:~/community-edition-setup/output# /opt/symas/bin/slapadd -n 1 -l scripts.ldif 
57f6a2d3 str2entry: entry -1 has no dn
slapadd: could not parse entry (line=1)
_                       0.00% eta    14s elapsed            none spd  21.8 k/s 
Closing DB...

Removeing the line version: 1 solves the above error
-------------------------------------------------------------------------------------------------------
root@openldap-testbed:~/community-edition-setup/output# /opt/symas/bin/slapadd -n 1 -l asimba.ldif 
57f6a3c4 str2entry: entry -1 has no dn
slapadd: could not parse entry (line=1)
_                       2.56% eta   none elapsed            none spd 240.3 k/s 
Closing DB...

Removeing the line version: 1 solves the above error
-------------------------------------------------------------------------------------------------------

28sept2016

- There are two ways to configure the openldap installation
  = static - using the slapd.conf
  = dynamic - using the cn=config directory

slapd.conf - Easier to setup and modify, but every change requires that the slapd server is restarted
cn=config  - Changes are difficult to perform, needs to be written in LDIF format, but dynamic and doesn't require restart. 

So cn=config is the way to go.

- There are two builds of openldap, by the distribution and the one by the Symas Systems
- The Symas systems use the slapd.conf method and not the cn=config method. The setup is very easy with just the necessary things put in slapd.conf,
but import keeps failing due to the non existent schema elements.
- Way around it
  = write the python script to produce a complete schema using the 101-ox.ldif, 100-user.ldif and the 77-custom-attributes.ldif
  = then use that schema as the base and then import all the old data. Should workout since this is almost the replication of a complete install.

- Catch
  = It is still unclear how custom attributes can be created when using the slapd.conf method.
  = If it is found that the custom attributes cannot be created, then slapd.conf system should move to the cn=config system.

Next Steps:
- Write a python script to combine all the schema files into gluu.schema, user.schema and custom-attributes.schema
- Prepare a slapd.conf file with all the above schemas included in them
- Try importing a complete installation LDIF data
- If successful, then figure out how to move the entire system to cn=config 

29sept2016

Today's Research:
-----------------
- convertSchema.py converts both the 101-ox.ldif and the 77-customAttributes.ldif files into gluu.schema and custom.schema files respectively
- two important conf files are needed for the configuration of the OpenLDAP system
  = symas-openldap.conf gives the system configuration values
  = slapd.conf does the directory server configurations
- the include directive in the slapd.conf can be used to include files on the go. Things like indexing can be configured this way and included.
- Multiple suffix lines can be given, and at least one is required for each database definition.
  Hence both o=gluu and o=site can live in the same DB and can be accessed by the same rootDN
- RootDN - The DN need not refer to an entry in this database or even in the directory.
  
Solution for Yesterday's problem:
---------------------------------
- Moving to the cn=config setup is simple
- use slaptest to generate the cn=config directory and pass the location of the cn=config directory to the ldap server deamon using the -F <location of cn=config>

Today's Situation
-----------------
- both the config files have been copied to the community-edition-setup/static/openldap/ directory.
- now to configure the setup values according to the requirements of gluu server and write the setup.py functions to do it
- have to be careful about the setting up of passwords for both the directory and the cn=config

Next Steps
----------
- Configure the slapd.conf and symas-openldap.conf files
- Write a setup_openldap.py for all the functions required for setting up openldap