Instead of random number generation, use this. It's a LITTLE more secure.

generatePassword.java

//Make static if going to be used static, but you might want to put the Random instance outside the class
public String generatePassword(int characterLimit){
  Random r = new Random(); //New Random instance
  String u = UUID.randomUUID().toString().replaceAll("-", ""); //Get random Base64 string from a random UUID
  return u.substring(0, r.getNetInt(characterLimit-3)+3); //Returns a substring of the UUID
}

Updated to fix the fourth comment.