Merge pull request #2 from RichInSQL/Dev

Merge Dev To Main #1

README.md

@@ -1 +1,85 @@
-# RichMonitoring
+# RichMonitoring
+
+RichMonitoring is a SQL Server Inventory soloution that takes information from a number of System Views & DMV's on a schedule you define and loads them into tables for querying and reporting.
+
+# What Is Collected
+
+RichMonitoring currently collects the following data points - as part of RichMonitoring **NOTHING** leaves your SQL Enviroment **EVER**
+
+- Information on Database Files 
+- A list of all databases on the instance
+- The size of each database on the instance
+- All of the SQL Agent Jobs on the instance
+- All of the logins on the instance, this covers both Windows & SQL Logins
+- All of the objects on the instance, this loops each database and collects each object individually 
+- All of the logins on the instance that are sysadmin
+- The RichMonitoring soloution also has JobHistory archiving built in allowing SQL Agent Job logs to be moved to an archive table inside RichMonitoring, this is fully configurable from ```Config.AppConfig```. 
+
+All of the above collection points are configurable, you can turn them on or off in the ```Config.Inventory``` table.
+
+# Installation 
+
+Installation is easy, run the ```Installation.sql``` script on your SQL Server and RichMonitoring will be installed. For the installation to suceed you will need the following permissions;
+
+- Create Databases
+- Create Objects
+- Delete Objects
+- Alter Objects
+- Insert into objects
+
+Note: A SQL Server Agent job will be created with no schedule attached, you will need to create a schedule to run at a frequency that fits with your needs for the collection to automatically work. 
+
+# Configuration
+
+When the soloution is installed, it is installed with the default settings. All of the collection points are inactive.
+
+### Enabling a collection point 
+
+1. Find which collection points you would like to enable 
+
+```SELECT * FROM RichMonitoring.Config.Inventory```
+
+2. Update Config.Inventory setting the collection points that you wish to use to active (replacing 1,2,3 with the ID of the collection points you want)
+
+```UPDATE RichMonitoring.Config.Inventory SET Active = 1 WHERE ID IN (1,2,3)```
+
+### Managing Data Retention
+
+The RichMonitoring soloution will delete any Inventoried datapoints that are (by default) older then 30 days. If you would like to keep data for longer than this you can 100% do that. 
+
+```UPDATE [RichMonitoring].[Config].[AppConfig] SET INTValue = -180 WHERE ID = 1```
+
+Replace -180 with the amount of days you would like to keep 
+
+# Running RichMonitoring Manually
+
+If you would like to run RichMonitoring manually simply run the following command
+
+```EXEC RichMonitoring.app.usp_RunInventory```
+
+This will start the Inventory collection.
+
+# Can I see what was run and when?
+
+Sure, RichMonitoring has a RunLog, you can query this to see what was run when, this table falls within the Data Retention as mentioned in the Configuration section of this article so data will be purged in line with that configuration. 
+
+```SELECT * FROM RichMonitoring.Inventory.RunLog```
+
+
+# FAQ
+
+- Q. Does RichMonitoring support Always On Availability groups?
+- A. At the moment RichMonitoring doesn't officially support Always On Availability groups, though this is something we want to add in the future. 
+
+- Q. Does RichMonitoring send any emails specific to the data collected?
+- A. No RichMonitoring doesn't send any emails, we do plan to add this feature in the future but when we do no sensitive data will ever leave your enviroment and most certainly won't be sent to RichInSQL.
+
+- Q. Do you have plans to add additional datapoints?
+- A. 100% Yes, after the initial release, RichInSQL plans to expand RichMonitoring to collect more information about your SQL Server.
+
+# How can we get help? 
+
+RichMonitoring comes as is for all non RichInSQL Clients, if you find a problem with RichMonitoring or it doesn't do something you wish it did, you have a few options available to you;
+
+1. Open a Issue right here on GitHub
+2. Fork the Repo, Fix the issue and open a pull request so we can improve RichMonitoring for others. 

src/Functions/App_DayOfWeekSunday.sql

@@ -0,0 +1,17 @@
+CREATE FUNCTION App.DayOfWeekSunday
+(
+	@Date DATETIME
+)
+RETURNS INT
+
+AS
+
+BEGIN 
+	DECLARE @DOW INT
+
+	SET @DOW = DATEPART(DW,@Date)
+	SET @Date = (@DOW + @@DATEFIRST - 1) % 7
+
+	RETURN @DOW + 1
+
+END

src/Schema/Security_App.sql

@@ -0,0 +1,3 @@
+CREATE SCHEMA [App]
+    AUTHORIZATION [dbo];
+

src/Schema/Security_Config.sql

@@ -0,0 +1,3 @@
+CREATE SCHEMA [Config]
+    AUTHORIZATION [dbo];
+

src/Schema/Security_Inventory.sql

@@ -0,0 +1,3 @@
+CREATE SCHEMA [Inventory]
+    AUTHORIZATION [dbo];
+

src/Stored Procedures/usp_ApplicationCleanup.sql

@@ -0,0 +1,76 @@
+CREATE PROCEDURE [App].[usp_ApplicationCleanup]
+
+AS
+
+SET NOCOUNT ON
+
+BEGIN 
+	BEGIN TRY
+		BEGIN TRANSACTION
+
+			DECLARE 
+				@Me VARCHAR(64),
+				@Days INT
+
+			SET @Me = CONCAT(OBJECT_SCHEMA_NAME(@@PROCID), '.',OBJECT_NAME(@@PROCID))
+			SET @Days = (SELECT INTValue FROM [Config].[AppConfig] WHERE ConfigName = 'Clean Up' AND Active = 1)
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Begin'
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting old records from DatabaseFiles'
+
+			DELETE FROM [Inventory].[DatabaseFiles] WHERE CensusDate < DATEADD(dd,@Days,GETDATE())
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting old records from Databases'
+
+			DELETE FROM [Inventory].[Databases] WHERE CensusDate < DATEADD(dd,@Days,GETDATE())
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting old records from DatabaseSize'
+
+			DELETE FROM [Inventory].[DatabaseSize] WHERE CensusDate < DATEADD(dd,@Days,GETDATE())
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting old records from Jobs'
+
+			DELETE FROM [Inventory].[Jobs] WHERE CensusDate < DATEADD(dd,@Days,GETDATE())
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting old records from Logins'
+
+			DELETE FROM [Inventory].[Logins] WHERE CensusDate < DATEADD(dd,@Days,GETDATE())
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting old records from Objects'
+
+			DELETE FROM [Inventory].[Objects] WHERE CensusDate < DATEADD(dd,@Days,GETDATE())
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting old records from RunLog'
+
+			DELETE FROM [Inventory].[RunLog] WHERE [EventDate] < DATEADD(dd,@Days,GETDATE())
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting old records from SysAdmins'
+
+			DELETE FROM [Inventory].[SysAdmins] WHERE CensusDate < DATEADD(dd,@Days,GETDATE())
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'End'
+
+		COMMIT TRANSACTION
+
+	END TRY
+	BEGIN CATCH
+	IF @@TRANCOUNT > 0 
+			ROLLBACK TRANSACTION
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'ERROR'
+
+			INSERT INTO App.SQL_Errors ([Username], [Error_Number], [ERROR_STATE], [ERROR_SEVERITY], [ERROR_LINE], [stored_Procedure], [ERROR_MESSAGE], [EventDate])
+			VALUES
+			  (
+			  SUSER_SNAME(),
+			   ERROR_NUMBER(),
+			   ERROR_STATE(),
+			   ERROR_SEVERITY(),
+			   ERROR_LINE(),
+			   ERROR_PROCEDURE(),
+			   ERROR_MESSAGE(),
+			   GETDATE()
+			   );
+	END CATCH
+END

src/Stored Procedures/usp_Cleanup_Job_History.sql

@@ -0,0 +1,159 @@
+
+CREATE PROCEDURE [App].[usp_Cleanup_Job_History]
+
+	AS
+
+	BEGIN
+
+		SET NOCOUNT ON;
+
+		--A couple of variables that we are going to use
+		DECLARE 
+			@Counter INT,
+			@MaxID INT,
+			@JobName NVARCHAR(128),
+			@Me VARCHAR(64) 
+
+		SET @Me = CONCAT(OBJECT_SCHEMA_NAME(@@PROCID), '.',OBJECT_NAME(@@PROCID))
+
+		EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Begin'
+
+		--A temp table to hold some results for us to loop through
+		CREATE TABLE #Results
+		(
+			ID INT IDENTITY(1,1) NOT NULL,
+			JobName NVARCHAR(128)
+		)
+
+		BEGIN
+
+			BEGIN TRY
+
+				BEGIN TRANSACTION t1
+
+					--Get the job name of the jobs we want to purge the data for
+					INSERT INTO #Results
+					SELECT 
+						[sJOB].[name] AS [JobName]
+					FROM
+						[msdb].[dbo].[sysjobs] AS [sJOB]
+
+						LEFT JOIN [msdb].[dbo].[sysjobschedules] AS [sJOBSCH]
+							ON [sJOB].[job_id] = [sJOBSCH].[job_id]
+
+						LEFT JOIN [msdb].[dbo].[sysschedules] AS [sSCH]
+							ON [sJOBSCH].[schedule_id] = [sSCH].[schedule_id]
+
+						LEFT JOIN [msdb].[dbo].[sysschedules] AS [SShed]
+							ON SShed.schedule_id = SSch.schedule_id
+
+						WHERE 
+						[sJOB].enabled = 1 --Make sure that the job is actually active
+						AND [sSCH].[schedule_uid] IS NOT NULL --Job is scheduled
+						AND [SShed].[freq_subday_type] = 4 --Jobs that run every x minutes
+						AND [SShed].[freq_subday_interval] < 60 --Jobs that run less than every 60 minutes
+
+				COMMIT TRANSACTION t1
+
+			END TRY
+			BEGIN CATCH 
+
+			ROLLBACK TRANSACTION t1
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'ERROR'
+
+			INSERT INTO App.SQL_Errors ([Username], [Error_Number], [ERROR_STATE], [ERROR_SEVERITY], [ERROR_LINE], [stored_Procedure], [ERROR_MESSAGE], [EventDate])
+			VALUES
+			  (
+			  SUSER_SNAME(),
+			   ERROR_NUMBER(),
+			   ERROR_STATE(),
+			   ERROR_SEVERITY(),
+			   ERROR_LINE(),
+			   ERROR_PROCEDURE(),
+			   ERROR_MESSAGE(),
+			   GETDATE()
+			   );
+
+			END CATCH
+
+		END
+
+		SET @Counter = 1
+
+		--Set the MAXID from the MAXID of the results table 
+		SET @MaxID = (SELECT MAX(ID) FROM #Results)
+
+		--If the counter is less or equal to the max id keep looping
+		WHILE @Counter <= @MaxID
+
+		BEGIN
+
+			BEGIN TRY
+
+				BEGIN TRANSACTION t2
+
+					--Get the job name from the results data set 
+					SET @JobName = (SELECT JobName FROM #Results WHERE ID = @Counter)
+
+					EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Archiving Job Data'
+
+					--Store the history of the data we are going to remove in a table that isn't MSDB
+					--that way we can add an index etc.
+					INSERT INTO [RichMonitoring].[dbo].[JobHistory_Archive](JobName,Message,Run_Date,Run_Time,Run_Status)
+
+					SELECT 
+						j.name,
+						jh.message,
+						jh.run_date,
+						jh.run_time,
+						jh.run_status 
+
+					FROM 
+						msdb..sysjobhistory jh
+					LEFT JOIN msdb..sysjobs j ON
+						j.job_id = jh.job_id
+
+					WHERE 
+					j.name = @JobName
+
+					EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'Deleting Job Data'
+
+					--Remove the data from the MSDB jobhistory table, we don't want any of this data keeping so we won't specify a date
+					EXEC msdb..sp_purge_jobhistory @job_name = @JobName
+
+				COMMIT TRANSACTION t2
+
+			END TRY
+			BEGIN CATCH
+
+			ROLLBACK TRANSACTION t2
+
+			EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'ERROR'
+
+			INSERT INTO App.SQL_Errors ([Username], [Error_Number], [ERROR_STATE], [ERROR_SEVERITY], [ERROR_LINE], [stored_Procedure], [ERROR_MESSAGE], [EventDate])
+			VALUES
+			  (
+			  SUSER_SNAME(),
+			   ERROR_NUMBER(),
+			   ERROR_STATE(),
+			   ERROR_SEVERITY(),
+			   ERROR_LINE(),
+			   ERROR_PROCEDURE(),
+			   ERROR_MESSAGE(),
+			   GETDATE()
+			   );
+
+			END CATCH
+
+			--Increment that counter and loop if we don't meet the MAX ID condition
+			SET @Counter = @Counter +1
+
+		END
+
+		--We are done, drop the temp table
+		DROP TABLE #Results
+
+		EXEC [App].[usp_InsertRunLog] @ProcedureName = @Me, @Action = 'End'
+
+	END