Browse Source

Merge branch 'develop' of lcmstaff/lcmlog-server into master

Wew, was this week long! Felt like almost two. As promised, merging with master.
Alessandro De Gennaro 4 years ago
parent
commit
ee6250f9bb
5 changed files with 396 additions and 42 deletions
  1. 1 0
      .gitignore
  2. 56 18
      README.md
  3. 15 0
      auth.toml
  4. 78 24
      lcmlog-server
  5. 246 0
      lcmlog-server-utils

+ 1 - 0
.gitignore

@@ -1,3 +1,4 @@
 data/
 auth/
 logs/
+auth_bkp/

+ 56 - 18
README.md

@@ -1,34 +1,72 @@
 # lcmlog-server
 
-Server side of a tool for reading and writing the lcm staff logs
-The program doen't accept any command line options.
-The first line of the input can be `GET`, `POST` or `UPDATE`.
+Server side of a tool for reading and writing the lcm staff logs.
 
-## The two kind of the logs
-To do
+- `lcmlog-server`: main script
+- `lcmlog-server-utils`: utility script
 
-## The authenticsation process
-To do
+## Input pipeline
+The first line is always the name of the method and can be only `GET`, `POST` or `UPDATE`.
 
-## The three methods
 ### GET
 The `GET` method is used to print all the logs that meet the required criteria.
-The second line of the input is the kind of the log to search for (see below), and it is the only mandatory parameter.
-The third line is the date: only logs whose date contains the searched for date are printed. The format for the date has not been decided yet, so currently any string can be inserted here.
-The fourth line is a list of comma separated tags to search for.
-The fifth line is the name of the user to search for.
+* The second line of the input is the type of log to search for (see below), and it is the only mandatory parameter.
+* The third line is the date: only logs whose date contains the searched date are printed.
+* The fourth line is a list of comma separated tags to search for.
+* The fifth line is the name of the user to search for.
 
 ### POST
 The `POST` method is used to add a log to the database.
-The second line of the input is the kind (se below).
-The third one is the date.
-The fourth one is a list of comma separated tags.
-The remainder of the file (until EOF is met) is the content of the log.
+* The second line of the input is the type (se below).
+* The third one is the date.
+* The fourth one is a list of comma separated tags.
+* The remainder of the file (until EOF is met) is the content of the log.
 
 ### UPDATE
 The `UPDATE` method is used to regenerate the internal server database.
 It should only be called if a log file is added by hand, or if some of the log files get renamed or corrupted.
 It assumes that all the log files are correctly formatted.
 
-## The format of the log files
-To do
+## Authentication groups
+There are four authentication groups with the following privileges:
+- 150: `GET 150`, `GET Admin` and `POST 150`.
+- Admin: `GET 150`, `GET Admin`, `POST 150`, `POST Admin` and `UPDATE`.
+- Nirvana (ex admin): `GET 150`, `GET Admin` and `POST Admin`.
+- Valhalla (ex 150): `GET 150` and `GET Admin`.
+
+## Initialize the environment
+The default directory is `/var/local/log/lcmlog-data`.
+
+### 1) Initialization of the folder structure
+Initialize the environment using `lcmlog-server-utils -i`. The structure is:
+- `auth`
+    - `auth.toml`
+    - `150`
+    - `Admin`
+    - `Nirvana`
+    - `Valhalla`
+- `data`
+    - `.data`
+- `logs`
+    - `logfile`
+
+### 2) Add users in authentication groups
+Edit files in `auth` folder for add users in a group. The format of a row is:
+```
+UID   username
+```
+
+### 3) Update ACL
+Authentication for read/write is based upon the access control list. To update ACL after _every_ changes in the `auth` folder, use `lcmlog-server-utils -u`.
+
+## Manage the environment
+- You can lookup authorizations using `lcmlog-server-utils -l`.
+
+- You can change authorizations by editing files in `auth` folder and then using `lcmlog-server-utils -u` to update ACL.
+
+- You can change the default privileges of an authentication groups by editing the `auth/auth.toml` file:
+    ```
+    ["example-group"]
+    auth=["priv-1","priv-2",...]
+    ```
+    The available privileges are: `GET 150`, `GET Admin`, `POST 150`, `POST Admin` and `UPDATE`. No further action is needed after editing this file.

+ 15 - 0
auth.toml

@@ -0,0 +1,15 @@
+# thanks to samuelecolombo (uid 17220) for toml
+
+["150"]
+auth=["GET 150","GET Admin","POST 150"]
+
+["Admin"]
+auth=["GET 150","GET Admin","POST 150","POST Admin","UPDATE"]
+
+# ex 150
+["Valhalla"]
+auth=["GET 150","GET Admin"]
+
+# ex admin
+["Nirvana"]
+auth=["GET 150","GET Admin","POST Admin"]

+ 78 - 24
lcmlog-server

@@ -1,15 +1,20 @@
 #!/usr/bin/env python3
 
-DIR="/usr/local/src/lcm-unimi/lcmlog-server"
+DIR = "/var/local/log/lcmlog-data"
 
 import os
 import os.path
+#from os import stat
+#from pwd import getpwuid
 import sys
 import pwd
 import logging
 import logging.handlers
 import hashlib
 import contextlib
+import toml
+import subprocess
+
 
 # We log what happens every time someone connects
 # Preparing the logger
@@ -23,8 +28,16 @@ file_handler = logging.handlers.TimedRotatingFileHandler(filename = DIR + "/logs
 file_handler.setFormatter(file_formatter)
 logger.addHandler(file_handler)
 
+# Update logfile acl
+#if pwd.getpwuid(os.stat(DIR + "/logs/logfile").st_uid).pw_name == pwd.getpwuid(os.geteuid()).pw_name:
+#    subprocess.call(["touch", DIR + "/logs/logfile"])
+#    #subprocess.call(["chmod", "444", DIR + "/logs/*"])
+#    subprocess.call(["chmod", "666", DIR + "/logs/logfile"])
+
+
 #------------------------------------------------------------------------------
 
+
 def main():
 
 	# The user is going to call us through ssh, so to know who he is we can simply get his effective uid
@@ -42,7 +55,8 @@ def main():
 		else:
 			kind = input()	# This is the kind of the log, and it can be 150 or Admin
 			logger.info("Kind: " + kind)
-			# TODO: check if the date format is valid (after deciding which format is to be considered valid)
+			# Now date check is implemented in the client side script
+			# Maybe, for the future, date control can be implemented aslo here
 			date = input()	# The date of the log
 			logger.info("Date: " + date)
 			tags = input()	# Tags are comma separated
@@ -51,7 +65,7 @@ def main():
 			if kind != "150" and kind != "Admin":	# We only have this two log types
 				raise KindError
 			if method == "POST":
-				auth(user_id, "POST", kind)	# Check if the user can post for the requested kind
+				auth(user_id, "POST", kind)     # Check if the user can post for the requested kind
 				log = sys.stdin.read() # Read the log content
 				method_post(kind, user_name, date, tags, log)
 			elif method == "GET":
@@ -90,15 +104,16 @@ def main():
 	finally:
 		logger.info("End\n")
 
+
 #------------------------------------------------------------------------------
 
+
 # Create new log file and adds it to the database
 # kind, user_name, date and tags is the log metadata
 # log is the log content
 # The log metadata and content is hashed, and the hash is saved in the database and used as the filename for the log
 # The return value of the function is the hash
 def log_create(kind, user_name, date, tags, log):
-	# TODO: utf-8 encoding doesn't allow accented characters. Find a more suitable encoding
 	name = hashlib.sha512((kind + user_name + date + tags + log).encode("utf-8")).hexdigest()
 	with open(DIR + "/data/" + name, "x") as f:
 		f.write(name + "\n" + kind + "\n" + user_name + "\n" + date + "\n" + tags + "\n" + log)	# Write the file
@@ -106,6 +121,7 @@ def log_create(kind, user_name, date, tags, log):
 		f.write(name + ":" + kind + ":" + user_name + ":" + date + ":" + tags + "\n")	# And add the entry to the .data file
 	return name
 
+
 # Search for the requested entry
 # The functions returns a list containing the hash (saved in the database) of all the files that meet the specified criteria
 # The kind parameter is mandatory (because different users have different privileges based on it).
@@ -116,21 +132,29 @@ def log_find(kind, user_name, date, tags):
 		for line in f:
 			found = True
 			l = line.split(":")
-			if l[1].find(kind) == -1:	# The kind is different
+			# The kind is different
+			if l[1].find(kind) == -1:
 				continue
-			if user_name and l[2].find(user_name) == -1:	# The username is different, or we aren't searching by username
+			# The username is different, or we aren't searching by username
+			if user_name and l[2].find(user_name) == -1:
 				continue
-			if date and l[3].find(date) == -1:	# The date is different, or we aren't searching by date
+			# The date is different, or we aren't searching by date
+			if date and l[3].find(date) == -1:
 				continue
+			# Searh tags
 			for t in tags.split(","):
 				if t and l[4].find(t) == -1:
 					found = False
 					break
+			# Save
 			if found:
 				file_list.append(l[0])
 	return file_list
 
-# TODO: the following functions work with the hash of the log files. The problem is that there are three different places where the hash is: the first line of the file, the database entry for the log and the filename of the log. I have to decide which function operates on which hash, because for example if the hash is changed in the file, it needs to be changed also in the other two locations.
+
+# TODO: the following functions work with the hash of the log files. The problem is that there are three different places where the hash is: the first line of the file,
+# the database entry for the log and the filename of the log. I have to decide which function operates on which hash, because for example if the hash is changed in the file,
+# it needs to be changed also in the other two locations.
 # Add log file to .data
 # This function reads an existing log file and adds it to the database
 # Tha hash that is saved in the database is not calculated: the first line in the file is considered to be the hash. Use log_check to check if they are the same
@@ -142,9 +166,11 @@ def log_add(name):
 			   f.readline().rstrip("\n") + ":" +	# Date
 			   f.readline().rstrip("\n") + "\n") 	# Tags
 
+
 # Check if the saved hash is correct, and if it is not, ask the user what to do
 # This function calculates the hash of the file with filename name, and returns True if it is the same as the first line of the file, False otherwise
-# If it doesn't correspond, it asks the user if he wants to keep it like it is or change it. Currently, it is pretty messed up: only the hash saved in the file is changed, not the one saved in the database or the file name. Also, the dialog to ask if the hash is to be changed probably should not be in this function.
+# If it doesn't correspond, it asks the user if he wants to keep it like it is or change it. Currently, it is pretty messed up: only the hash saved in the file is changed,
+# not the one saved in the database or the file name. Also, the dialog to ask if the hash is to be changed probably should not be in this function.
 def log_check(name):
 	with open(DIR + "/data/" + name, "r") as f:
 		saved_hash = f.readline().rstrip("\n")
@@ -177,13 +203,13 @@ def log_check(name):
 				logger.info("Hash changed")
 				break
 	return result
-	
+
 
 # Calculates hash of file
 # The first line of the file is the saved hash, therefore it is not considered in the calculation
 def log_hash(name):
 	with open(DIR + "/data/" + name, "r") as f:
-		f.readline()	# The saved hash doesn'e enter in the calculation
+		f.readline()	# The saved hash doesn't enter in the calculation
 		kind = f.readline().rstrip("\n")
 		user_name = f.readline().rstrip("\n")
 		date = f.readline().rstrip("\n")
@@ -191,8 +217,10 @@ def log_hash(name):
 		log = f.read()
 		return hashlib.sha512((kind + user_name + date + tags + log).encode("utf-8")).hexdigest()
 
+
 #------------------------------------------------------------------------------
 
+
 # Print specified log on stdout
 def method_get(kind, user_to_find, date, tags):
 	file_list = log_find(kind, user_to_find, date, tags)
@@ -204,38 +232,60 @@ def method_get(kind, user_to_find, date, tags):
 			sys.stdout.write("Date: " + f.readline())
 			sys.stdout.write("Tags: " + f.readline())
 			sys.stdout.write("\n" + f.read() + "------------------\n")
-	logger.info("GET succesful: got " + str(len(file_list)) + " files")
+	logger.info("GET successful: got " + str(len(file_list)) + " files")
+
 
 # Write log
 def method_post(kind, user_name, date, tags, log):
 	name = log_create(kind, user_name, date, tags, log)
-	logger.info("POST succesful: hash " + name)
+	logger.info("POST successful: hash " + name)
+
 
 # Generate .data file
 def method_update():
 	with contextlib.suppress(FileNotFoundError):
-		os.remove(DIR + "/data/.data")
+	    os.remove(DIR + "/data/.data")
 	file_list = os.listdir(DIR + "/data/")
 	open(DIR + "/data/.data", "x").close()
 	for name in file_list:
-		newname = log_hash(name)
-		if not log_check(name):
-			os.rename(DIR + "/data/" + name, DIR + "/data/" + newname)
-		log_add(newname)
-	logger.info("UPDATE succesful: added " + str(len(file_list)) + " files")
+            newname = log_hash(name)
+            if not log_check(name):
+                os.rename(DIR + "/data/" + name, DIR + "/data/" + newname)
+            log_add(newname)
+	logger.info("UPDATE successful: added " + str(len(file_list)) + " files")
+
 
 #------------------------------------------------------------------------------
 
+
 # Checks if the user has the permissions to use the requested method
 def auth(user_id, method, kind):
-	with open(DIR + "/auth/" + kind + "/" + method) as f:
-		for line in f:
-			if int(line) == user_id:
-				return
-	raise AuthError()
+	# Check if user id is in auth files
+	# We suppose that every authorized user is ONLY IN A FILE!
+	user_type = ""
+	for file_name in ["150","Admin","Valhalla","Nirvana"]:
+		with open(DIR + "/auth/" + file_name) as f:
+			for line in f:
+				line = line.split()[0]
+				if int(line) == user_id:
+					# If present, we consider only the user type
+					user_type = file_name
+					break
+			if not user_type == "":
+				break
+	else:
+		# If not in auth files, the user cannot do anything
+		raise AuthError()
+	# Now we check the user type permissions
+	auth_list = toml.load(DIR + "/auth/auth.toml")[user_type]["auth"]
+	if not method + " " + kind in auth_list:
+		raise AuthError()
+	return
+
 
 #------------------------------------------------------------------------------
 
+
 # Error definitions
 class AuthError(Exception):
 	pass
@@ -244,8 +294,12 @@ class KindError(Exception):
 class MethodError(Exception):
 	pass
 
+
 #------------------------------------------------------------------------------
 
+
 # Starting point
 if __name__ == "__main__":
 	main()
+	# Change permissions to logfile just before leaving. Dirty fix to a not well understood problem
+	subprocess.call(["chmod", "666", DIR + "/logs/logfile"])

+ 246 - 0
lcmlog-server-utils

@@ -0,0 +1,246 @@
+#!/bin/bash
+
+
+DIR="/var/local/log/lcmlog-data"
+
+
+######################
+# AUXILIARY FUNCTION #
+######################
+
+
+usage()
+{
+	echo "Usage: $0 [-i|-l|-u]"
+	echo "Use -h option to show the help message."
+}
+
+
+######
+
+
+method_help()
+{
+	echo "lcmlog-auth-utils: utility for manage the authorization system of lcmlog-server."
+	echo
+	echo "Usage: $0 [option]"
+	echo
+	echo "Options:"
+	echo "  -i  Initialization of the folder structure in default directory ($DIR)"
+	echo "  -l  See who is authorized in $DIR/auth"
+	echo "  -u  Update folders ACL (data and logs)"
+	echo
+	echo "  -h  Show this help"
+}
+
+
+######
+
+
+method_init()
+{
+	if ! [ -d $DIR ]; then
+		echo "Folder $DIR not present: you should create it first!"
+		exit 0
+	else
+		# auth folder
+		if [ -d $DIR/auth ]; then
+			echo "Folder $DIR/auth already present. Exit."
+		else
+			echo "Setting up auth folder..."
+			mkdir $DIR/auth
+			touch $DIR/auth/150
+			touch $DIR/auth/Admin
+			touch $DIR/auth/Valhalla # ex 150
+			touch $DIR/auth/Nirvana  # ex admin
+			cp /usr/local/src/lcm-unimi/lcmlog-server/auth.toml $DIR/auth/auth.toml
+			echo "Completed."
+		fi
+
+		# data folder
+		if [ -d $DIR/data ]; then
+			echo "Folder $DIR/data already present. Exit."
+		else
+			echo "Setting up data folder..."
+			mkdir $DIR/data
+			touch $DIR/data/.data
+			echo "Completed."
+		fi
+
+		# logs folder
+		if [ -d $DIR/logs ]; then
+			echo "Folder $DIR/logs already present. Exit."
+		else
+			echo "Setting up logs folder..."
+			mkdir $DIR/logs
+			touch $DIR/logs/logfile
+			echo "Completed."
+		fi
+
+		echo
+		echo "Now you should add entries in $DIR/auth files and then update folder permissions using -u flag."
+
+	fi
+}
+
+
+######
+
+
+method_authlookup()
+{
+	if [ -d $DIR/auth ]; then
+		
+		echo
+		echo -e "\033[1;36mAUTHORIZATIONS in $DIR/auth:\033[00m"
+		echo
+		
+		# look 150
+		echo -e "\033[01m150 auth: GET 150, GET Admin, POST 150\033[00m"
+		for uid in $(awk '{print $1}' $DIR/auth/150); do
+			getent passwd $uid
+		done
+		echo
+
+		# look admin
+		echo -e "\033[01mAdmin auth: GET 150, GET Admin, POST 150, POST Admin, UPDATE\033[00m"
+		for uid in $(awk '{print $1}' $DIR/auth/Admin); do
+			getent passwd $uid
+		done
+		echo
+
+		# look ex 150
+		echo -e "\033[01mValhalla auth: GET 150, GET Admin\033[00m"
+		for uid in $(awk '{print $1}' $DIR/auth/Valhalla); do
+			getent passwd $uid
+		done
+		echo
+
+		# look ex admin
+		echo -e "\033[01mNirvana auth: GET 150, GET Admin, POST Admin\033[00m"
+		for uid in $(awk '{print $1}' $DIR/auth/Nirvana); do
+			getent passwd $uid
+		done
+		echo
+		echo
+		
+		# look dir permissions
+		echo -e "\033[1;36mACL PERMISSIONS:\033[00m"
+		echo
+		getfacl $DIR/data
+		getfacl $DIR/data/.data
+		getfacl $DIR/logs
+
+	else
+		echo "auth folder not present. Use the -i option to create it."
+	fi
+}
+
+
+######
+
+
+method_updateACL()
+{
+	if [ -d $DIR/auth ] && [ -d $DIR/data ] && [ -d $DIR/logs ]; then
+
+		method_authlookup
+
+		echo "Do you want to set new permissions? [y|n]"		
+		read ANSWER
+        while [ -z $ANSWER ]; do
+                echo "Yes (y) or no (n)? "
+                read ANSWER
+        done
+        while [ $ANSWER != "y" ] && [ $ANSWER != "n" ]; do
+                echo "Yes (y) or no (n)? "
+                read ANSWER
+                while [ -z $ANSWER ]; do
+                        echo  "Yes (y) or no (n)? "
+                        read ANSWER
+                done
+        done
+        if [ $ANSWER == "n" ]; then exit 0; fi
+        echo
+
+		# remove old permissions
+		echo "Removing old permissions..."
+		setfacl -b $DIR/data
+		setfacl -b $DIR/data/.data
+		setfacl -b $DIR/logs
+
+		# add new permissions
+		echo "Setting up new permissions..."
+
+		# set 150
+		for uid in $(awk '{print $1}' $DIR/auth/150); do
+			setfacl -m u:$uid:rwx $DIR/data
+			setfacl -m u:$uid:rw $DIR/data/.data
+			setfacl -m u:$uid:rwx $DIR/logs
+		done
+
+		# set admin
+		for uid in $(awk '{print $1}' $DIR/auth/Admin); do
+			setfacl -m u:$uid:rwx $DIR/data
+			setfacl -m u:$uid:rw $DIR/data/.data
+			setfacl -m u:$uid:rwx $DIR/logs
+		done
+		
+		# set ex 150
+		for uid in $(awk '{print $1}' $DIR/auth/Valhalla); do
+			setfacl -m u:$uid:rx $DIR/data
+			setfacl -m u:$uid:r $DIR/data/.data
+			setfacl -m u:$uid:rwx $DIR/logs
+		done
+		
+		# set ex admin
+		for uid in $(awk '{print $1}' $DIR/auth/Nirvana); do
+			setfacl -m u:$uid:rwx $DIR/data
+			setfacl -m u:$uid:rw $DIR/data/.data
+			setfacl -m u:$uid:rwx $DIR/logs
+		done
+
+		echo "Done."
+		echo
+		echo "New permissions are:"
+		method_authlookup
+		
+	else
+		echo "Error: some folders are missing. Use the -i option to create missing folders."
+	fi
+}
+
+
+###############
+# MAIN SCRIPT #
+###############
+
+
+## ENTRY POINT - INITIALIZATION
+
+# Check number of arguments
+if (( $# != 1 )); then
+	usage
+	exit 1
+fi
+
+# Check that the option is only one
+# -->   -i   OK
+#       -il  NO
+if (( ${#1} != 2 )); then
+	usage
+	exit 1
+fi
+
+# Select method from the option
+while getopts ":iluh" o; do
+	case $o in
+		i) method_init;;
+		l) method_authlookup;;
+		u) method_updateACL;;
+		h) method_help;;
+		\?) usage && exit 1;;
+	esac
+done
+
+exit 0