Skip to content

Instantly share code, notes, and snippets.

@smothiki

smothiki/readme.md

Last active October 9, 2017 21:08
Show Gist options
  • Save smothiki/59cf8c0a750e8c2c8b3dc90fd9a6c25d to your computer and use it in GitHub Desktop.
Save smothiki/59cf8c0a750e8c2c8b3dc90fd9a6c25d to your computer and use it in GitHub Desktop.
Download ZIP
weebly docs
Raw
readme.md

File Sharing App weebly trial week project.

  • basic requirement are that a User can upload, Update, Share and delete files.

User Model

type User struct {
	UserID   string   `json:"user_id"`
	PassWord string   `json:"password"`
	Files    []string `json:"files"`
	Shared   []string `json:"shared_files"`
}

The user model matches the user table in the Database

desc user
username text PRIMARY KEY,
password text,
file list<text>,
sharedfiles list<text>,

  • Every user is unique and identified by user name. User has a password to authenticate. A list of files in file column to identify the files Owned by the user. Sharedfiles is a list of shared files shared by other users.

  • When a User X shares a file A to User Y we append X&A file to shared files to identify that a file A is shared by user X.

Files model.

type File struct {
	Filename string   `json:"file_name"`
	Username string   `json:"user_name"`
	Users    []string `json:"users"`
	Version  int      `json:"version"`
	Data     []byte   `json:"data"`
}

The File model matches the file table in the database

filename text,
username text,
users list <text>,
data blob,
version int,
PRIMARY KEY (username, filename))

File is uniquely identified by filename and username data is a blob to store the file bytes. Whenever there is an update to the file we increment the version. users is a list of users who share the file. username is the owner of the file.

##APIS

  • "/", indexPageHandler : This is used to login to the applicaiton
  • /internal", internalPageHandler : handler that is called after login and shows the user name and logout form.
  • "/upload", upload : upload is a GET call to fetch the upload form to upload a file.
  • "/update", update : This is similar to upload but fetched the same form to PUT operation.
  • "/deletefiles", deletefiles: GET operation to Get the form that deletes files.
  • "/deleteuser", deleteuser: GET operation to Get the form that deletes files.
  • "/api/v1/user/{user}/file/{file}: getfileofuser. is a Get operation that get the specified file that belongs to the user. If file is not found return file not found error.
  • "/api/v1/user/{user}", listallfilesofuser : This is a get operation to list all files of the user.
  • "/api/v1/user/{user}/share", sharefile : Share file is a POST operation. That takes a map of list of string. Key in the map is a file and value is a list of usernames that file has to be shared with We iterate over each file key and appends the user in the users list of each file. Then each file is appended to the shared file list of each user in the value.
  • "/api/v1/user/{user}", fileupload : File upload is a post and put operation POST operation if the file is not present. Put operation to update an existing file.
  • "/api/v1/user", deleteuser: POST operation only permitted by admin user. This is supposed to be DELETE method but the form DELETE actions are throwing some issues. For convenience of this peoject made this POST call.
  • "/api/v1/user/{user}/files", deletefiles: "POST" operation to delete a list of files from the user. Again it should be a DELETE call for convenience purposes made it post. This deletes the file from user table file list and entry from file table along with shared file entries from other users.
  • "/signin", signinHandler: Used to signin a new user.
  • "/login", loginHandler: creates a session and stores it in as a cookie and uses that cookie for authentication.
  • /logout", logoutHandler: "POST" operation but clears the cookie.

Developer doc

  • Run a local cassandra instance docker run --name cassandra -p 9042:9042 -d cassandra:3
  • Set the following ENV variables KEYSPACE and DBENDPOINT(127.0.0.1)
  • Now Set the publisher by setting TOPIC and PROJECT env vars. make sure that topic and project exists in GCP. Set GOOGLE_APPLICATION_CREDENTIALS to the path to the service account.
  • Run dbinti job go run dbinit.go. This should initialize the Database with tables and keyspace.
  • go run webserver/server.go should start your server after Initializing the cassandra session.

Subscriber

  • Subscriber listens to the events published by the weeblyserver. Requires TOPIC, SUBSCRIPTION, PROJECT env vars to start the subscriber. go run subscriber/subscriber.go should start the subscriber.

Deploying the APP into kubernetes cluster.

  • helm install --namespace "cassandra" -n "cassandra" incubator/cassandra This will install a cassandra 3 node cluster as a stateful set.
  • Fetch Ips of the stateful set kubectl get pods -o yaml --namespace=cassandra|grep podIP: and update the DBENDPOINT in the manifests/dbinitpod.yaml, weebly_res.yaml.
  • Deploy weebly_service.yaml which is a service of type LoadBalancer.
  • Deploy servcesecret.yaml which has b64 encoded service account file.
  • Deploy dbinitpod.yaml which is a job that Initialises database and user.
  • deploy weebly_res.yaml which is a replica set of the server.

Building code

Make docker-build will build subscriber dbinitpod and server linux binaries and builds docker container. which can be used to deploy in kubernetes cluster.

Understanding code

  • webserver folder has the code to start the file server application.
  • models folder has the user model that talks to database.
  • pkg has user package that acts as a interface between user model and web server.
  • manifests folder has all the manifests that are required to deploy the applicaiton.
  • subscriber has the code for subscriber that receive messages.
  • publisher has the publisher class that sends messages.
  • rootfs is where all the dockerfile and binaries go to.
  • database folder has all the info related to database folder.

TODO

  • Developed some unit tests. But this project requires functional tests than writing a unitest by mocking DB.
  • Better documentation in the code explaining more functional details of a method.
  • Helm chart for the deployment if pending.
  • Better error handling and session handling other than just storing the session in the browser. Or might be using OAUTH type of Auth.
  • Better messaging to pubsub.

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment