sync

OCTPDBA-380 - POC data anonymization with Data Masker by RedGate tool/analyze.txt

@@ -1,3 +1,26 @@
+Review functionalities
+    => done
+    
+Find DevOps implementation examples
+    => not many, but done
+    
+Define how to anonymize data between interconnected environments
+    => see below
+    
+Identify interesting functionalities for DBA team in TriaPharm context
+
+Test interesting functionalities
+
+Evaluate Licensing costs for TriaPharm Integration/Test environment
+    => done, see SWOT page
+Document on Confluence (link to Red-gate documentation when it is more efficient)
+    => done, see SWOT page
+SWOT
+ => done: https://galenica.atlassian.net/wiki/spaces/DBA/pages/402555774/Data+masking+using+Red-Gate+Data+Masker
+ 
+Present to DBA team
+
+
 ressources:
     https://www.red-gate.com/hub/university/courses/product-training?tool=data-masker&level=get-started
         getting started videos from red-gate directly
@@ -7,6 +30,9 @@ ressources:
     https://learn.microsoft.com/en-us/sql/relational-databases/security/dynamic-data-masking?view=sql-server-ver16
         sql server >= 2016 implémente aussi du data masking
         
+    https://en.wikipedia.org/wiki/Data_masking
+    
+    https://www.red-gate.com/blog/audit-and-compliance/masking-your-on-premise-database-with-sql-data-mask
     
 sites intéressants:
     https://plantbasedsql.com/tag/data-masking/
@@ -17,6 +43,23 @@ bcp d'exemples font usage de red-gate sql data catalog
     
     SQL Data Catalog allows users to catalog their SQL Server data estate by applying classifications, as tags and free-text labels, to SQL Server objects. The taxonomy of tags and attributes to be applied is also created and managed by this product. A common use case for the tool is for classifying columns by their sensitivity under data privacy regulations such as the GDPR.
     
+    https://www.red-gate.com/hub/university/courses/data-masker/getting-started-with-data-masker-for-sql-server/masking-rules-and-concepts/datasets-can-i-make
+    parle de la création de son propre dataset, et de dataset correllés
+    il serait possible de créer un dataset correlé depuis le ref adresse, et de l'utiliser pour le maskage des pharmacies,
+    !!!!!!!!!!!!!!!!! 
+    Il existe une rêgle de syncro qui permet d'utiliser une table d'une autre db pour peupler la table locale.
+    Si l'on masque les données du ref adresse, on peut réutiliser ces données avec cette syncro pour la redescendre dans la pharmacie a masquer.
+    A voir si il est possible de syncroniser dans l'autre sens, soit depuis data masker, la réplication ou si un ETL devrait être mis en place depuis la pharmacie vers le ref adresse 
+    => oui, il est possible de pousser depuis data masker via une "cross database table to table syncronisation rule"
+    !!!!!!!!!!!!!!!!!
+    
+    
+    !!!!!!!!!!!!!!!!!
+    https://www.red-gate.com/hub/university/courses/data-masker/advanced-operations-with-data-masker/advanced-operations-with-data-masker/synchronizing-between-databases
+    comment masquer entre plusieurs db
+    !!!!!!!!!!!!!!!!!
+
+    
 et peut-être plus intéressant est le package de "SQL provision":
     SQL Provision is a solution for (compliant) test data management that combines two Redgate products into a single offering: Data Masker for static data masking, and SQL Clone for database cloning and provisioning. 
     
@@ -26,7 +69,30 @@ terminologie:
     Deterministic data masking is the process of masking data with values in a repeatable way, such that it will give the same value when masked in any and all future runs on any value that matches and will create a new record for values which have not been previously masked. An example of this would be if you were to mask “Chris Unwin” to “Brad Pitt”, it should appear as “Brad Pitt” not only in our (for example) dbo.Contacts table but also all associated tables (regardless of PKFK relationships at the DB level) and every single run should provide the same output. This is useful for building up familiarity with the data and utilizing for future test runs.
     
 dans notre cas, c'est du deterministic qu'il faudra utiliser.
-Pour ce faire, il faut:
+https://plantbasedsql.com/2019/12/03/deterministic-data-masking-the-who-who-and-who-and-how/
+==> non, un static data masking est utilisable, mais avec des rêgles de syncronisation entre les tables et db
+
+Pour démarrer:
+https://plantbasedsql.com/2020/01/07/where-do-i-even-begin-with-data-masking-getting-started-in-3-steps/
 
 1. Identifier les champs et les tables qui sont à masquer
-2. définir les règles de masquage
+    red-gate sql catalog est prévu pour ça.
+    
+    There should be a record, tagged with a reasonable value to indicate at the very least the following 4 things:
+        What system it is used in   => le produit
+        Who is in charge            => l'application owner
+        How sensitive it is         => a déterminer
+        What kind of data it is     => a déterminer
+        
+        Retention ?
+        treatment intent (encrypt, masking) ?
+        
+2. définir les règles de masquage
+
+caveats
+    data masker ne peut pas travailler en substitution sur une table qui n'a ni PK ni unique constraint.
+    Il utilise ces infos pour reconnaitre le champ d'origine
+    
+    les indexes peuvent ralentir le masking. renvoi vers une video de red-gate university
+    
+    

OCTPDBA-431 - Spike create numeric Pharmacode column to improve joins/check data.sql

@@ -49,3 +49,20 @@ AND [ik].[ITK_key]=@phidx
 ORDER BY [ik].[ITK_key]
 ;
 
+return
+SELECT 
+      [i].[Item_ID]
+    ,[it].[ITTX_description]
+    ,[it].[ITTX_language]
+    ,[ik].[Item_key_ID]
+    ,[ik].[ITK_key]
+    ,[ik].[ITK_label_text]
+    ,[ik].[ITK_subsidiary]
+    ,[pi].*
+FROM [dbo].[Item] [i]
+    JOIN [dbo].[Item_key] [ik] ON ik.[ITK_item] = i.[Item_ID]
+    JOIN [dbo].[PH_item] [pi] ON [pi].[PHIT_item] = i.[Item_ID]
+    LEFT JOIN [dbo].[Item_text] [it] ON it.[ITTX_item] = i.[Item_ID] AND [it].[ITTX_language] = 1
+WHERE [ik].[ITK_type] = 1 --phcode
+ORDER BY [ik].[ITK_key]
+;