Disable and Enable Subscriptions in Reporting Services

Data Quality is very important and you want to make sure the business users get the Correct data from the Cube or from the Datawarehouses. In some circumstances where the data loads may fail you may want to disable the subscriptions so that the business users dont get the wrong reports. I use the following script to disable the reporting services subscriptions

UPDATE J SET
	J.Enabled = 0
FROM sysjobs J
INNER JOIN syscategories C
ON J.category_id = C.category_id
WHERE C.[Name] = 'Report Server'

You can add the above script as a job step if the BI Load fails.

SQL Service Accounts and Maintenance

At sometime in the afternoon today, I was alerted by the business that SQL reporting services is down. The first thing I checked was to open a report. I saw the following error message on the browser.


The report server has encountered a configuration error. Logon failed for the unattended execution account. (rsServerConfigurationError)
Log on failed. (rsLogonFailed)
The report server cannot open a connection to the report server database. A connection to the database is required for all requests and processing. (rsInternalError)
Login failed. The login is from an untrusted domain and cannot be used with Windows authentication.

I did not understand this as I have checked the same report 10 minutes ago. So checked the databases and all the connections. Everything seems fine. Now checked the application logs on the server and the errors came up indicating the service accounts access is denied. I was surprised and asked the System Administrator if the permissions were changed on the Active Directory account. The permissions seems to be fine but for some reason the account is locked out. Upon investigation I realised that one of the administrators was trying to use the SQL service account for installing a new SQL server box and typed the password wrong as many a times to make the lockout happen.

Lessons Learnt
a. create a seperate SQL server account for each Live SQL Server Instance
b. Make sure you dont type the password wrong for the Service accounts

SQL SERVER :Date and Time Dimensions in T-SQL using CTE

Everytime we create a cube for analysis services Date and Time Dimensions are needed to slice and dice the historical Data. Sql Server Analysis Services uses the Time Intelligence Features to populate the date and time. Its posted in detail in this blog.

However we need to use them in our relational Databases to do some adhoc reporting. I always use the following CTE’s to create Date and Time Dimensions. The good thing about CTE is you can recursively loop through the data till you get the desired result.

DATE DIMENSION :

I got the Date Dimension idea originally from JamieT (Jamie Thomson,MVP SQL Server) ‘s blog.

with DateCTE  as
(
	select cast ('01-Jan-2009' as datetime)   Datevalue
	union all
	select datevalue + 1
	from DateCTE where datevalue + 1 < = '31-Dec-2020'
)
select cast(datevalue as int ) as DateID
,datename(year,DateValue) as [Year]
,dateName(Month,DateValue) as [Month]
,datename(d,datevalue) as [datenumber]
,datename(Week,Datevalue) as [Week]
,datename(DW,Datevalue) as [Day]
,Datevalue  as [Date]
from DateCTE
order by Datevalue
option (maxrecursion 0)

TIME DIMENSION :

  with HourCTE(Hour,Minute) as
  (
  select 0 as Hour , 0 as Minute
  union all
  select Hour , Minute + 1
  from HourCTE
  where Minute + 1 < 60
  ),
   MinuteCTE(Hour,minute) as
  (
    select Hour, Minute from HourCTE
	union all
	select Hour + 1 ,Minute from MinuteCTE
	where Hour + 1 < 24
)
select * from minuteCTE
order by Hour ,Minute 

Running SQL Reporting Services Report from SSIS

Whenever you create a new Subscription in Sql server reporting services, a new job is created under the Sql agent as shown in the figure below.

You can use the msdb.dbo.sp_start_job procedure to start the job. In SSIS create a new SQL task and run the job wherever required

exec msdb.dbo.sp_start_job @job_name = 'JobName'

For more information about sp_start_job please refer MSDN

Could not open global shared memory to communicate with performance DLL; data flow performance counters are not available. To resolve, run this package as an administrator, or on the system’s console.

You may see the above error if you are using the SQL server Destination in your SSIS package.

Sql server Destination cannot be used in packages that access a SQL Server database on a remote server. Instead use the OLEDB destination.

Things to note for SQL Server Destination

a. It offers the same speed as the Bulk Insert Operation.
b. It does not support Error Output.

For more information please check msdn

Happy New Year in TSQL

I did use SQL String manipulations a lot this year. I want to show an example of how to use CHAR.

Open your Management Studio – and paste the following in your Query Window

SELECT CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15) + CHAR(15) + CHAR(15) + CHAR(15) + CHAR(13)
+ CHAR(72) + CHAR(65) + CHAR(80) + CHAR(80)+ CHAR(89) + ' ' + CHAR(78) + CHAR(69)+CHAR(87)+ ' ' + CHAR(89)+ CHAR(69)+ CHAR(65) + CHAR(82)
+CHAR(13)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15)+CHAR(15) + CHAR(15) + CHAR(15) + CHAR(15)

Run the Query(Try Pressing Ctrl-T and run the Query again).

Want to know what each CHAR value represent ?
Run the following:
 

DECLARE @i int
SET @i = 1
WHILE @i < = 256 Begin PRINT CHAR(@i) + ' --> ' + CONVERT(VARCHAR,@i)
SET @i = @i + 1
END

Enjoy!!!

Wish you all a Happy New Year.

@VivekSerou

Reporting Services ToolTips

One good feature in SQL server Reporting Services is ToolTip . It is a simple feature but has more advantages if you want to show the complete information in a report. Its a great way of saving some Reporting space.I use it when Designing Dashboards or if you want add the redundant columns like the Description fields.

Here is an example from AdventureWorks.

SQL script

SELECT
	pp.ProductLine,
	pp.ProductNumber,
	pp.Name ,
        pp.Class ,
        pp.Color ,
        pp.FinishedGoodsFlag ,
        pp.ListPrice ,
        pp.Style ,
        pd.Description AS [Description] ,
        psc.Name AS SubCategoryName ,
        pc.Name AS CategoryName
        FROM    Production.ProductDescription pd
        JOIN Production.ProductModelProductDescriptionCulture pmpdc
				ON pd.ProductDescriptionID = pmpdc.ProductDescriptionID
        JOIN Production.Product pp ON pp.ProductModelID = pmpdc.ProductModelID
        JOIN Production.ProductSubcategory psc
				 ON psc.ProductSubcategoryID = pp.ProductSubcategoryID
        JOIN Production.ProductCategory pc
				 ON pc.ProductCategoryID = psc.ProductCategoryID
        WHERE pmpdc.CultureID = 'en'
ORDER BY pp.Name

Report Layout

ReportingServicesLayout

I have not included the Category and SubCategory field in the above report . Instead I have included this as ToolTip

a. Right Click on the ProductNumber Text box
b. TextBox Properties –> ToolTip

ShowTooltip

I added the following expression for Category and SubCategory

CategoryExpression

So when you hover over the ProductID, the end result is

ShowCategories

You can do the same thing for large descriptive fields

ShowDesc

Creating a simple FrontEnd using Reporting Services and Merge

Problem:
A Business User wants to update/insert/delete the values in a table that sits in the metadata of the Datawarehouse on a regular basis. Based on these values numerous calculations will be done in the warehouse which generates a data driven Report on a weekly basis.

Initially I got the following Solutions
1. Create a frontend app for the user to populate the values (the most common Global solution)
2. Ask the user to populate the values in an excel file with the columns in specific format so that I can use my SSIS to do the lookup with the original table and update/delete/insert based on the values.( But there are lot of things to consider when using excel like Problems with incorrect format of excel files and maintaining them.)

So when I was reviewing the Merge feature in TSQL 2008, an idea stuck in mind

The solution I ended up doing was

Creating a Reporting services Report that takes the values as parameter and use the MERGE statement to insert/update/delete the values in the backend table

Here is the Example of what I mean:

create table CricketRuns
(
PlayerID int identity(1,1),
PlayerName nvarchar(max),
Runs int,
Average float
)
go

insert into CricketRuns
select 'Sachin Tendulkar',17000,45.23
union
select 'Ricky Ponting',12000,41.08
union
select 'Kevin Peiterson',4000,46.00
union
select 'Chris Gayle',7000,34.87
union
select 'Jacques Kallis',11000,42.09
union
select 'Ross Taylor',2500,41.00
go
 

So the table looks like

InitialResults

I created this procedure using the Merge Statement that will insert / delete / update based on the players name

create procedure uspInsertRuns
	(
		@PlayerName nvarchar(max)
		,@Runs int
		,@Average float
		,@status nvarchar(100)
	)
		as
	Merge into CricketRuns as Target
	using
		(select @PlayerName,@Runs,@Average,@status) as source(Playername,runs,Average,Status)
	on
		(Target.Playername = source.PlayerName)
		when matched and Source.Status = 'D' then
		Delete
		when matched then
		update set runs = source.runs , Average = source.Average
		when not matched then
		insert (PlayerName,runs,Average)  values (source.PlayerName,source.runs,source.Average);

		select PlayerName,Runs,Average from CricketRuns order by PlayerID
 

So you can insert,delete or update as below

exec uspInsertRuns ‘Andrew Strauss’,2000,35,” –> Insert
exec uspInsertRuns ‘Virendar Shewag’,5000,37,” –> Insert
exec uspInsertRuns ‘Ross Taylor’,0,0,’D’ –> Delete
exec uspInsertRuns ‘Sachin Tendulkar’,20000,45,” –> Update

Just add the proc in a reporting services and expose it to the business user and thats it..A frontend using Reporting Services
ReportingServices

So without any excel management or creating a .net frontend reporting services has done the job.

List of Temp Tables in your SQL server

select create_date,modify_date,OBJECT_NAME(object_id) as TableName
from tempdb.sys.objects
where name like ‘%#%’

Encryption in SQL SERVER using SSIS

I have been recently working on an Sql Server Encryption Project. It includes getting the data from a source and importing them into the Datawarehouse via the ETL(SSIS in this case) and then Encrypting it. I decided to blog on Encryption to share my experience.

a. When you Encrypt the Data in SSIS make sure you Encrypt in Batches(i.e dont encrypt the same data again).
b. The datatypes should be valid for the Encrypted Content and should have enough space to store it(I used nvarchar(max))

Below is a small Example of the Encryption Process:

Encrypting the Database

use EncryptionTest

--Create a Master Key by Encryption
create master key Encryption by password = 'Sql5erver'

--Create the Certificate
create certificate testcert with Subject = 'Testing the Symmetric Encryption'

--Create the Symmetric Key with Algorithm and Encrypt it with the certificate
create Symmetric Key testkey with algorithm =  AES_256 encryption by certificate testcert;

Sample Tables and Data

create Database Encryptiontest
go

create table Source
(
	UserID int
	,[Name] varchar(100)
	,Password varchar(max)
)
go

insert into Source
select 1,'Vivek','Sqlserver007'
union
select 2,'Jermey','MungingData'
union
select 3,'Marcus','SirMarcus'
union
select 4,'Cesare','Berlusconi'
go

create table EncryptedData
(
	userID int
	,[Name] varchar(100)
	,EncryptedPassword nvarchar(max)
	,Logid int
)

go

Procedure for Encryption

The Detailed Syntax for EncryptbyKey is given in this MSDN article

create procedure EncryptData
(
	@Logid int
)

as

Open Symmetric Key Testkey Decryption by Certificate TestCert

update EncryptedData
set EncryptedPassword = ENCRYPTBYKEY(Key_guid('testkey'),EncryptedPassword) where Logid = @Logid

Now Coming to the SSIS Task. I have used the Logid as the variable which controls the records to be encrypted in every flow(i.e the Log id changes for every iteration of the dataflow task). Once the records are transferred to the Destination table EncryptedData the procedure Encryptdata @Logid is executed.

SSISTask

Variables and the DataSourcetask

Encryption_DataFlow

DataFlow task and Derived Column

DataFlow_DerivedColumn

How to Decrypt the Data?

Once the Data is encrypted the table is displayed as follows

EncryptedData

For Decryption the End user should have the following permissions
Grant Control on Certificate
Grant View Definition on Symmetric Key

Following is the way to Decrypt. You need to open the Symmetric key to Decrypt it.

Open Symmetric Key TestKey Decryption by Certificate TestCert

select UserID,[name],CAST(DecryptByKey(EncryptedPassword) as nvarchar) as Password from
EncryptedData
 

The following are the DMV’s(Dynamic Management Views) that can be used for Encryption

--Checking the Symmetric keys
select * From sys.symmetric_keys

--Checking the certificates
select  * from sys.certificates

--Checking for any openkeys
select * From sys.openkeys

There are some good articles on Encryption that may be useful to you in future.

Encryption HierArchy
http://blogs.technet.com/keithcombs/archive/2005/11/24/sql-server-2005-data-encryption.aspx