tag:blogger.com,1999:blog-77983772024-03-07T06:16:52.151-08:00Sanjay kattimani's technical blogsanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.comBlogger70125tag:blogger.com,1999:blog-7798377.post-53017330624235409892020-09-12T12:03:00.023-07:002021-09-19T08:17:44.185-07:00Dynamic Data Masking feature of Snowflake<p><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXDhKrAzUrnmf8u2JdSSqCB4zisVLqDDba_MjTkuJF_0LnS4ypljA7YB0DQClJBPxCKQ3mq81Zeca7CmFhiZUId-lPjZeYNqVHHjW8fCmK2GjM3y0vZKPKbgBFp0Ovq2dJnhKjHQ/s1140/dynamic-data-masking-snowflake.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="722" data-original-width="1140" height="254" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXDhKrAzUrnmf8u2JdSSqCB4zisVLqDDba_MjTkuJF_0LnS4ypljA7YB0DQClJBPxCKQ3mq81Zeca7CmFhiZUId-lPjZeYNqVHHjW8fCmK2GjM3y0vZKPKbgBFp0Ovq2dJnhKjHQ/w400-h254/dynamic-data-masking-snowflake.png" width="400" /></a></div><p> <span style="font-size: medium;"> Data security over cloud is a concern for many organizations while moving their data to cloud. With newer and ever growing regulations and laws on data privacy and protections, it is ever more challenging landscape for organizations to protect their customer's data.<br /></span></p><p><span style="font-size: medium;"> <span> </span><span> </span>In July 2020 Snowflake announced availability of new set of column level security features available called dynamic data masking. This lets organizations control the exposure of Personally Identifiable Information (PII), Protected Health Information (PHI). There are a number of security products such as <a href="https://www.youtube.com/watch?v=A_ta4dL9ezo" target="_blank">SecuPi</a> provide these features as their product services, Now snowflake offers its own built in feature with many advantages compared to third party products. This features offers<br /></span></p><ul style="text-align: left;"><li><span style="font-size: medium;">Ease of use</span></li><li><span style="font-size: medium;">No additional performance hit</span></li><li><span style="font-size: medium;">Seamless integration with rest of snowflake's features & roles</span></li><li><span style="font-size: medium;">No additional cost</span></li></ul>Dynamic Data Masking is one the new feature introduced in Snowflake to cater to data security and privacy both from external and internal users.<br /><br /><h2 style="text-align: left;">Creating an effective data protection policy</h2><div>In order to create an effective data masking and protection policy we need to orchestrate policies through roles instead of individual users. Following steps can be created to do the same.</div><div style="text-align: left;"><br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfKAdqeXo0ZzpPcIWgSKm7kkY8trvHk1yuF1oEvzN7sCPuizd5fssbSE_HaKfEGsfwH53CURsaw1KxfhbzghkOR1vGyo4e9GRWQ-3knRgsB3iEelXYUHBzNCAnlo1PPeZw27SApw/s2656/dynamic-data-masking-steps-to-applly.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1142" data-original-width="2656" height="172" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjfKAdqeXo0ZzpPcIWgSKm7kkY8trvHk1yuF1oEvzN7sCPuizd5fssbSE_HaKfEGsfwH53CURsaw1KxfhbzghkOR1vGyo4e9GRWQ-3knRgsB3iEelXYUHBzNCAnlo1PPeZw27SApw/w400-h172/dynamic-data-masking-steps-to-applly.png" width="400" /></a></div><br /><h4 style="text-align: left;">Create Role</h4><p>Commands to create a user, a role. You could also manage these by </p><table class="highlight tab-size js-file-line-container" data-paste-markdown-skip="" data-tab-size="8" style="background-color: white; border-collapse: collapse; border-spacing: 0px; color: #24292e; font-size: 14px; tab-size: 8;"><tbody style="box-sizing: border-box;"><tr style="box-sizing: border-box;"><td class="blob-code blob-code-inner js-file-line" id="LC7" style="box-sizing: border-box; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"><table class="highlight tab-size js-file-line-container" data-paste-markdown-skip="" data-tab-size="8" style="border-collapse: collapse; border-spacing: 0px; color: #24292e; font-size: 14px; tab-size: 8;"><tbody style="box-sizing: border-box;"><tr style="box-sizing: border-box;"><td class="blob-code blob-code-inner js-file-line" id="LC4" style="box-sizing: border-box; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"><span class="pl-k" style="box-sizing: border-box; color: #d73a49;">Create</span> <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">user</span> <span class="pl-en" style="box-sizing: border-box; color: #6f42c1;">PII_USER</span> password<span class="pl-k" style="box-sizing: border-box; color: #d73a49;">=</span>piiuser
default_role <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">=</span> masking_admin must_change_password <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">=</span> FALSE;</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="5" id="L5" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: left; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td></tr></tbody></table> Create Role masking_admin;</td></tr></tbody></table><h4>Assign Role<br /><p style="text-align: left;"><span style="font-weight: normal;">Now assign the data masking role to a user who can mask the data and define all rules around how data should be presented to different roles or users.</span></p></h4><table class="highlight tab-size js-file-line-container" data-paste-markdown-skip="" data-tab-size="8" style="background-color: white; border-collapse: collapse; border-spacing: 0px; color: #24292e; font-size: 14px; tab-size: 8;"><tbody style="box-sizing: border-box;"><tr style="box-sizing: border-box;"></tr><tr style="box-sizing: border-box;"><td class="blob-code blob-code-inner js-file-line" id="LC9" style="box-sizing: border-box; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"><span class="pl-k" style="box-sizing: border-box; color: #d73a49;">grant</span> role masking_admin to user PII_USER;</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="10" id="L10" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: left; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td></tr></tbody></table><h4 style="text-align: left;">Apply Policy</h4><p>You can assign masking policies at DB object levels, in this example I am applying to account and a schema.</p><table class="highlight tab-size js-file-line-container" data-paste-markdown-skip="" data-tab-size="8" style="background-color: white; border-collapse: collapse; border-spacing: 0px; color: #24292e; font-size: 14px; tab-size: 8;"><tbody style="box-sizing: border-box;"><tr style="box-sizing: border-box;"><td class="blob-code blob-code-inner js-file-line" id="LC22" style="box-sizing: border-box; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top;"><span face="SFMono-Regular, Consolas, Liberation Mono, Menlo, monospace"><span style="font-size: 12px; white-space: pre;">grant apply masking policy on account to role masking_admin; </span></span></td></tr></tbody></table><h4 style="text-align: left;">Create Policy</h4><p><span class="pl-k" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace" style="background-color: white; box-sizing: border-box; color: #d73a49; font-size: 12px; white-space: pre;">grant</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace" style="background-color: white; color: #24292e; font-size: 12px; white-space: pre;"> create masking policy </span><span class="pl-k" face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace" style="background-color: white; box-sizing: border-box; color: #d73a49; font-size: 12px; white-space: pre;">on</span><span face="SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace" style="background-color: white; color: #24292e; font-size: 12px; white-space: pre;"> schema STAGE to role masking_admin;</span></p><h4 style="text-align: left;">Apply policy to columns</h4><div>Now we create a masking policy for function or UDF that implements masking of data. In this first example we return the original Social security number if the user's role is Masking_Admin, returns a masked data for everyone other role including account admins.</div><div>In the second example of masking email ids, if the user belongs to ACCOUNTADMIN role, email id is partially masked and domain name part is returned as is and rest of the details of email id are masked with a function.</div><div>Finally we need to apply these masking policies to individual columns as needed. You can apply these policies to thousands of columns and they will all mask the data for different roles as expected </div><div></div><table class="highlight tab-size js-file-line-container" data-paste-markdown-skip="" data-tab-size="8" style="background-color: white; border-collapse: collapse; border-spacing: 0px; color: #24292e; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji"; font-size: 14px; tab-size: 2;"><tbody style="box-sizing: border-box;"><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="3" id="L3" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC3" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"><span class="pl-c" style="box-sizing: border-box; color: #6a737d;"><span class="pl-c" style="box-sizing: border-box;">--</span>Create a new policy</span></td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="4" id="L4" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC4" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">
</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="5" id="L5" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC5" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">create <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">or</span> replace masking policy <span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">STAGE</span>.<span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">SSN_Policy</span> <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">as</span> (SSN string) returns string <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">-</span><span class="pl-k" style="box-sizing: border-box; color: #d73a49;">></span></td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="6" id="L6" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC6" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">case when current_role() <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">in</span> (<span class="pl-s" style="box-sizing: border-box; color: #032f62;"><span class="pl-pds" style="box-sizing: border-box;">'</span>MASKING_ADMIN<span class="pl-pds" style="box-sizing: border-box;">'</span></span>) then</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="7" id="L7" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC7" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"> SSN</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="8" id="L8" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC8" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"> ELSE</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="9" id="L9" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC9" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"> <span class="pl-s" style="box-sizing: border-box; color: #032f62;"><span class="pl-pds" style="box-sizing: border-box;">'</span>**Masked SSN**<span class="pl-pds" style="box-sizing: border-box;">'</span></span></td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="10" id="L10" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC10" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">END;</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="11" id="L11" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC11" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">
</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="12" id="L12" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC12" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">
</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="13" id="L13" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC13" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"><span class="pl-c" style="box-sizing: border-box; color: #6a737d;"><span class="pl-c" style="box-sizing: border-box;">--</span>-Associate it</span></td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="14" id="L14" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC14" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"><span class="pl-k" style="box-sizing: border-box; color: #d73a49;">alter</span> <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">table</span> customer modify column ssn <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">set</span> masking policy <span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">Stage</span>.<span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">SSN_Policy</span>;</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="15" id="L15" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC15" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">
</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="16" id="L16" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC16" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"><span class="pl-k" style="box-sizing: border-box; color: #d73a49;">SELECT</span> <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">*</span> <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">FROm</span> <span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">stage</span>.<span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">CUSTOMER</span>;</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="17" id="L17" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC17" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">
</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="18" id="L18" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 2px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC18" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 2px; position: relative; vertical-align: top; white-space: pre;">create <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">or</span> replace masking policy <span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">STAGE</span>.<span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">Email_Policy</span> <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">as</span> (email string) returns string <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">-</span><span class="pl-k" style="box-sizing: border-box; color: #d73a49;">></span></td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="19" id="L19" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 2px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC19" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 2px; position: relative; vertical-align: top; white-space: pre;">case when current_role() <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">in</span> (<span class="pl-s" style="box-sizing: border-box; color: #032f62;"><span class="pl-pds" style="box-sizing: border-box;">'</span>MASKING_ADMIN<span class="pl-pds" style="box-sizing: border-box;">'</span></span>) then email</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="20" id="L20" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC20" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"> when current_role() <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">in</span> (<span class="pl-s" style="box-sizing: border-box; color: #032f62;"><span class="pl-pds" style="box-sizing: border-box;">'</span>ACCOUNTADMIN<span class="pl-pds" style="box-sizing: border-box;">'</span></span>) then regexp_replace(email,<span class="pl-s" style="box-sizing: border-box; color: #032f62;"><span class="pl-pds" style="box-sizing: border-box;">'</span>.+<span class="pl-cce" style="box-sizing: border-box;">\@</span><span class="pl-pds" style="box-sizing: border-box;">'</span></span>,<span class="pl-s" style="box-sizing: border-box; color: #032f62;"><span class="pl-pds" style="box-sizing: border-box;">'</span>*****@<span class="pl-pds" style="box-sizing: border-box;">'</span></span>)</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="21" id="L21" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC21" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"> else <span class="pl-s" style="box-sizing: border-box; color: #032f62;"><span class="pl-pds" style="box-sizing: border-box;">'</span>**Masked Email**<span class="pl-pds" style="box-sizing: border-box;">'</span></span></td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="22" id="L22" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC22" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">END;</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="23" id="L23" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC23" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">
</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="24" id="L24" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC24" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;">
</td></tr><tr style="box-sizing: border-box;"><td class="blob-num js-line-number" data-line-number="25" id="L25" style="box-sizing: border-box; color: rgba(27, 31, 35, 0.3); cursor: pointer; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; min-width: 50px; padding: 0px 10px; text-align: right; user-select: none; vertical-align: top; white-space: nowrap; width: 50px;"></td><td class="blob-code blob-code-inner js-file-line" id="LC25" style="box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, monospace; font-size: 12px; line-height: 20px; overflow-wrap: normal; overflow: visible; padding: 0px 10px; position: relative; vertical-align: top; white-space: pre;"><span class="pl-k" style="box-sizing: border-box; color: #d73a49;">alter</span> <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">table</span> customer modify column emailid <span class="pl-k" style="box-sizing: border-box; color: #d73a49;">set</span> masking policy <span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">Stage</span>.<span class="pl-c1" style="box-sizing: border-box; color: #005cc5;">Email_Policy</span>;</td></tr></tbody></table><div style="text-align: left;"><br /></div><p>When implemented with right set of policies, visibility of your data can be controlled at a granular level.<br /><br />Check out my YouTube video highlighting the feature in detail with a live demo<br /></p><div class="separator" style="clear: both; text-align: left;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/Utlvqs8y6xo" width="460" youtube-src-id="Utlvqs8y6xo"></iframe></div><br /><p><br /></p><h3 style="text-align: left;">3rd Party tools</h3><p>There are a number third party security offerings such as <a href="https://www.youtube.com/watch?v=A_ta4dL9ezo&list=PLUxVA4dlJgcz90Vbufh6o6p8Rrew-lnE3&index=5" target="_blank">SecuPi</a> that provide overarching security suites that cater to many other tools of your IT ecosystem, but Dynamic data masking feature is a built in feature that does not have any negative impact or security concerns or additional installation requirements.</p><h4 style="text-align: left;">Conclusion</h4><p>Another great feature by snowflake that sets is a leader that addresses every aspect of cloud data ware housing needs.</p></div>sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-7766628900278957852019-03-16T20:36:00.000-07:002019-03-16T20:37:29.768-07:00Snowflake Data Sharing - Complete video tutorialSnowflake's data sharing feature lets you share your data with other account holders. Since data is shared as readonly, consumers do not incur any charges but they only pay for the compute resource they use.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEig03SXPChwDfrQYwkX8maikVrdHIP6M0A4kAdI_tyl-9IGPe1_i3wW2qnH_nTGKcrq5g3zHom6_7Y7LEzYvvhHrCGNtIoTntE_sie3WwM8-_S0HmtUYGWmaUFt6sQtWTNqL44tgg/s1600/datasharing-snowflake-architecture.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="snowflake-data-sharing" border="0" data-original-height="895" data-original-width="1600" height="358" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEig03SXPChwDfrQYwkX8maikVrdHIP6M0A4kAdI_tyl-9IGPe1_i3wW2qnH_nTGKcrq5g3zHom6_7Y7LEzYvvhHrCGNtIoTntE_sie3WwM8-_S0HmtUYGWmaUFt6sQtWTNqL44tgg/s640/datasharing-snowflake-architecture.png" title="snowflake-data-sharing" width="640" /></a></div>
<br />
Checkout the detailed instructions video on my channel<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/GcrEK-VFEB8/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/GcrEK-VFEB8?feature=player_embedded" width="320"></iframe></div>
<br />
<br />
LInk to this video - <a href="https://www.youtube.com/watch?v=GcrEK-VFEB8&list=PLUxVA4dlJgcz90Vbufh6o6p8Rrew-lnE3">https://www.youtube.com/watch?v=GcrEK-VFEB8&list=PLUxVA4dlJgcz90Vbufh6o6p8Rrew-lnE3</a><br />
<br />sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-49467677408309502082019-03-16T20:25:00.003-07:002019-03-16T20:25:54.921-07:00Snowflake Secure ViewsSnowflake's secure views let you abstract the logic and hide the underlying tables structures and their relations. These secure views help you manage data security and privacy.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieGgkZVPzwBtm3XdrXva8tKSqnvWMnb10qa5YCTW8NX-ZKxlbwsfLx0O-9DxaAwGEKMXxJwx4Pr8r_DfhvNm31zrNMN1najlb0Vo5Sq8hyPwIyzyje4KgNudk-9MpczHajC9PsZQ/s1600/snowflake-secure-views.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="snowflake-secure_views" border="0" data-original-height="952" data-original-width="1260" height="241" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieGgkZVPzwBtm3XdrXva8tKSqnvWMnb10qa5YCTW8NX-ZKxlbwsfLx0O-9DxaAwGEKMXxJwx4Pr8r_DfhvNm31zrNMN1najlb0Vo5Sq8hyPwIyzyje4KgNudk-9MpczHajC9PsZQ/s320/snowflake-secure-views.png" title="snowflake-secure_views" width="320" /></a></div>
<br />
In this video i reviewed all the details related to secure views and demo of using them.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/CRZQjxP10G0/0.jpg" src="https://www.youtube.com/embed/CRZQjxP10G0?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<br />
<br />
Commands used in the video to demonstrate snowflake's secure views. Checkout the detailed video on my channel - <a href="https://www.youtube.com/watch?v=CRZQjxP10G0&list=PLUxVA4dlJgcz90Vbufh6o6p8Rrew-lnE3&index=3">https://www.youtube.com/watch?v=CRZQjxP10G0&list=PLUxVA4dlJgcz90Vbufh6o6p8Rrew-lnE3&index=3</a><br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">--LIST PATIENTS</span><br />
<span style="font-family: Courier New, Courier, monospace;">SELECT * FROM PATIENTS;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">CREATE OR REPLACE SECURE VIEW ORG_PATIENTS AS</span><br />
<span style="font-family: Courier New, Courier, monospace;">SELECT * FROM PATIENTS ;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">SELECT * FROM ORG_PATIENTS ;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">SHOW VIEWS;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">SELECT FULLNAME,CITY,USER_ACCESS.ORG_ID,USER_ACCESS.ACCOUNT_ID FROM patients </span><br />
<span style="font-family: Courier New, Courier, monospace;">INNER JOIN USER_ACCESS ON PATIENTS.ORG_ID=USER_ACCESS.ORG_ID;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">insert into USER_ACCESS values ('Baylor Hospital','');</span><br />
<span style="font-family: Courier New, Courier, monospace;">insert into USER_ACCESS values ('Mayo Clinic','');</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">create table USER_ACCESS</span><br />
<span style="font-family: Courier New, Courier, monospace;">(</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ORG_ID VARCHAR(50),</span><br />
<span style="font-family: Courier New, Courier, monospace;"> ACCOUNT_ID varchar(50)</span><br />
<span style="font-family: Courier New, Courier, monospace;">);</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">insert into USER_ACCESS values ('Baylor Hospital','MN30689');</span><br />
<span style="font-family: Courier New, Courier, monospace;">insert into USER_ACCESS values ('Mayo Clinic','PO51568');</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">SELECT * FROM USER_ACCESS; </span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">TRUNCATE TABLE USER_ACCESS SET ACCOUNT_ID='PO51568' WHERE </span><br />
<span style="font-family: Courier New, Courier, monospace;">ORG_ID='Mayo Clinic';</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">SELECT FULLNAME,CITY,USER_ACCESS.ORG_ID,USER_ACCESS.ACCOUNT_ID FROM patients </span><br />
<span style="font-family: Courier New, Courier, monospace;">INNER JOIN USER_ACCESS ON PATIENTS.ORG_ID=USER_ACCESS.ORG_ID;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">select CURRENT_ACCOUNT();</span><br />
<span style="font-family: Courier New, Courier, monospace;">select CURRENT_ROLE ();</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">--CREATE A SECURE VIEW</span><br />
<span style="font-family: Courier New, Courier, monospace;">CREATE OR REPLACE SECURE VIEW ORG_PATIENTS AS</span><br />
<span style="font-family: Courier New, Courier, monospace;">SELECT FULLNAME,CITY,USER_ACCESS.ORG_ID FROM patients </span><br />
<span style="font-family: Courier New, Courier, monospace;">INNER JOIN USER_ACCESS ON PATIENTS.ORG_ID=USER_ACCESS.ORG_ID</span><br />
<span style="font-family: Courier New, Courier, monospace;">WHERE ACCOUNT_ID=CURRENT_ACCOUNT();</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">select * from ORG_PATIENTS;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">SHOW VIEWS;</span><br />
<span style="font-family: Courier New, Courier, monospace;">describe view ORG_PATIENTS;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">describe view mycustomerview;</span><br />
<span style="font-family: Courier New, Courier, monospace;">describe view ORG_PATIENTS;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">select get_ddl('view', 'ORG_PATIENTS') view_defn;</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<br />
<br />
<br />
<br />sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-38401636159140800302019-03-16T20:11:00.000-07:002019-03-16T20:27:55.346-07:00Snowflake Time Travel - A complete guide<!--StartFragment-->
<br />
<div style="font-family: Calibri; font-size: 11.0pt; margin: 0in;">
<br /></div>
<div style="font-size: 11pt; margin: 0in;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">In the recent years
IT Roles such as Database administrators are becoming less and less relevant as most cloud based
databases and datawarehouses are managed by IT engineers needing very little or
no administration.</span></div>
<div style="font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-size: 11pt; margin: 0in;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Snowflake already
has very little administration. You do not need to do typical DBA tasks like index
tables, partitioning or deal with space related issues. Time travel is another
feature that lets developers self serve dba related task of reverting or restoring
the database after you do issue some inadvertent commands.</span></div>
<div style="margin: 0in;">
<span style="font-size: 14.6667px;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /><iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/IK1LwL_Xl_4/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/IK1LwL_Xl_4?feature=player_embedded" width="320"></iframe></span></div>
<div style="margin: 0in;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="margin: 0in;">
<span style="font-size: 14.6667px;"><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /></span></span></div>
<div style="font-size: 11pt; margin: 0in;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">In this detailed hands on video I explained about parameters that affect time travel and how to utilize this feature to revert the accidental or intentional database changes. </span></div>
<div style="font-size: 11pt; margin: 0in;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Here is the video link - <a href="https://www.youtube.com/watch?v=CRZQjxP10G0&list=PLUxVA4dlJgcz90Vbufh6o6p8Rrew-lnE3&index=3">https://www.youtube.com/watch?v=CRZQjxP10G0&list=PLUxVA4dlJgcz90Vbufh6o6p8Rrew-lnE3&index=3</a></span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div style="font-size: 11pt; margin: 0in;">
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Here are the list of commands used in the video</span></div>
<blockquote class="tr_bq" style="margin: 0in;">
<span style="font-family: "courier new" , "courier" , monospace;"><br /><span style="font-size: 14.6667px;">-- DATA_RETENTION_TIME_IN_DAYS</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">show parameters;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">show parameters in database SALES;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">show parameters in warehouse XMALLFORFINANCE;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">show parameters in account;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">show parameters like '%DATA%';</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">show parameters like 'DATA%' in account;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">--CHANGES DATA RETENTION FOR THE DATABASE</span><span style="font-size: 14.6667px;">show parameters in database SALES;</span><span style="font-size: 14.6667px;">alter DATABASE SALES set DATA_RETENTION_TIME_IN_DAYS = 1 ;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">--CHANGES DATA RETENTION FOR THE WHOLE ACCOUNT</span><span style="font-size: 14.6667px;">show parameters like 'DATA%' in account;</span><span style="font-size: 14.6667px;">alter ACCOUNT set DATA_RETENTION_TIME_IN_DAYS = 1 ;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">select * from PATIENTS;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">INSERT INTO PATIENTS values (5,'TEST PATIENT','AMSTERDAM','Mayo Clinic');</span><span style="font-size: 14.6667px;">select * from PATIENTS at(offset => -60*5);</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">delete from PATIENTS where id=5;</span><span style="font-size: 14.6667px;">drop table sales.snow.PATIENTS ;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">--Over a day</span><span style="font-size: 14.6667px;">select * from PATIENTS at(offset => -60*500000);</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">select * from PATIENTS at(timestamp => 'Sat, 16 Mar 2019 16:20:00 -0700'::timestamp);</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">select * from user_access;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">--Before changes make by a specific query</span><span style="font-size: 14.6667px;">select * from patients before(statement => '025e545d-fc23-4e8d-9ac5-335943a1bec2');</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">create table restored_table clone user_access</span><span style="font-size: 14.6667px;"> at(timestamp => 'Mon, 09 May 2019 01:01:00 +0300'::timestamp);</span><span style="font-size: 14.6667px;"> </span><span style="font-size: 14.6667px;"> create table restored_table clone user_access</span><span style="font-size: 14.6667px;"> at(offset => -60*10);</span><span style="font-size: 14.6667px;"> </span><span style="font-size: 14.6667px;"> select * from restored_table;</span><span style="font-size: 14.6667px;"> drop table restored_table;</span><span style="font-size: 14.6667px;"> </span><span style="font-size: 14.6667px;"> --Schema as existed 1 hour before</span><span style="font-size: 14.6667px;"> create schema restored_schema clone snow at(offset => -3600);</span><span style="font-size: 14.6667px;"> drop schema restored_schema;</span><span style="font-size: 14.6667px;"> </span><span style="font-size: 14.6667px;"> create database restored_db clone sales</span><span style="font-size: 14.6667px;"> before(statement => '025e545d-fc23-4e8d-9ac5-335943a1bec2');</span><span style="font-size: 14.6667px;"> drop database restored_db;</span><span style="font-size: 14.6667px;"> </span><span style="font-size: 14.6667px;">show tables history in sales.snow;</span><span style="font-size: 14.6667px;">show tables history in sales;</span><span style="font-size: 14.6667px;">---Restoring objects</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">undrop table patients;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">undrop schema snow;</span><span style="font-size: 14.6667px;"><br /></span><span style="font-size: 14.6667px;">undrop database sales;</span></span></blockquote>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div style="font-size: 11pt; margin: 0in;">
<br /></div>
<div style="font-size: 11pt; margin: 0in;">
<br /></div>
<!--EndFragment--><span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">More details are available in snowflake's documentation page - <a href="https://docs.snowflake.net/manuals/user-guide/data-time-travel.html">https://docs.snowflake.net/manuals/user-guide/data-time-travel.html</a></span><br />
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "helvetica neue" , "arial" , "helvetica" , sans-serif;">Check out my youtube channel for more videos - https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ</span><br />
<br />sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-32299390752105991732018-04-12T05:13:00.000-07:002018-04-12T05:16:02.171-07:00Amazon S3 (Simple Storage Service) - object storage serviceOne of the Amazon's core service offering was highly scalable S3 storage. In this video i explained about its quick overview and hands on explanation most of its features including Security, versioning, static website hosting, accelerated transfers, event handling, tags etc.<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/dZmsfl9RlGk/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/dZmsfl9RlGk?feature=player_embedded" width="480"></iframe></div>
<div>
I covered this topic as I intend to do more videos in future that integrate with such services of cloud computing. Let me know your feedback.</div>
sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com2Dallas, TX, USA32.7766642 -96.79698789999997632.3496592 -97.442434899999981 33.2036692 -96.151540899999972tag:blogger.com,1999:blog-7798377.post-19880447600117052282017-05-03T21:15:00.004-07:002017-05-03T21:19:51.382-07:00Creating Azure Sql server database Microsoft's products have always been known for having very user-friendly interfaces. Azure is not different. Seemingly complicated PaaS, IaS and SaaS are all available under one Microsoft Azure umbrella. In this video of <a href="https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ" target="_blank">BigData Technologies</a>, I explore creating a new cloud computing based Azure Sql server database and then connect it to most common tools SSMS and Visual studio.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/xRcpsu1auVM/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/xRcpsu1auVM?feature=player_embedded" width="480"></iframe></div>
<br />
<br />
<b>Code used in the video</b><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">Create table cpt_codes(cptcode int, cpt_desc varchar(2000));</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: "verdana" , sans-serif;"><br /></span> <span style="font-family: Courier New, Courier, monospace;">insert into cpt_codes values (62320,'Injection(s), of diagnostic or therapeutic substance(s) (e.g.,anesthetic, antispasmodic, opioid, steroid, other solution),not including neurolytic substances, including needle orcatheter placement, interlaminar epidural or subarachnoid,cervical or thoracic; without imaging guidance');</span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="font-family: "verdana" , sans-serif;"><br /></span> <span style="font-family: Courier New, Courier, monospace;">select * from cpt_codes;</span></span><br />
<br />
As always, please feel free to comment on this blog or the video, and I will be happy to answer.sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-67102578306723714252017-02-05T04:40:00.000-08:002017-02-05T04:41:04.276-08:00MongoDB - A NoSQL, Json, RockMongo, RoboMongoMongoDB's popularity has been soaring in the recent years. The first video explores the details of Mongo DB, a sample document/table is created and we also see how to insert a sample Json data and extracting the it.<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/M_kcmQVh3c8/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/M_kcmQVh3c8?feature=player_embedded" width="480"></iframe></div>
The second video explores a few popular tools, their installation, and usage. RockMongo and RoboMongo are explored as an example.<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/dcAFSul78sE/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/dcAFSul78sE?feature=player_embedded" width="480"></iframe></div>
<br />sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com1tag:blogger.com,1999:blog-7798377.post-15120812039484557312017-02-05T04:31:00.000-08:002017-02-05T04:32:09.184-08:00Extracting HL7 data using Talend and storing in CassandraHL7 is a set of international standards used by clinical and healthcare providers to exchange information. This video explores using this HL7 data, extracting a few sample fields from it and then saving them into one of the popular NoSql database Cassandra.<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/j69L9poNXXg/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/j69L9poNXXg?feature=player_embedded" width="480"></iframe></div>
<br />sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com1tag:blogger.com,1999:blog-7798377.post-19074248855454336772017-02-05T04:19:00.004-08:002017-02-05T04:22:52.147-08:00<div class="separator" style="clear: both; text-align: left;">
Route53 of Amazon webservices is used for all hosting/Routing related needs. This video explores various options for setting up your domain name servers, how to redirect it to AWS and other settings inside AWS to make it redirect to your content on AWS. </div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCBvO2oYQwOJDA6Ij2jdXljTOBij55qzwTbiRYg9Yy2SoklvjiwF9D6u9_cVlZtNNUUPGECgepm-RX3q9GC-gtunHWbBD_opchfmGt-AHrpKXqTxf7TBSLsEavIAn38TNRaEJSqw/s1600/route53.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCBvO2oYQwOJDA6Ij2jdXljTOBij55qzwTbiRYg9Yy2SoklvjiwF9D6u9_cVlZtNNUUPGECgepm-RX3q9GC-gtunHWbBD_opchfmGt-AHrpKXqTxf7TBSLsEavIAn38TNRaEJSqw/s400/route53.png" width="400" /></a></div>
In this example i am demonstrating how to redirect your domain to a simple html page stored on AWS, process is the same even if you are redirecting to any app hosted on Amazon web servervices.<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/wQ4kiTasfNI/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/wQ4kiTasfNI?feature=player_embedded" width="480"></iframe></div>
<br />sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-35457947858556589532017-01-17T16:05:00.000-08:002017-01-17T16:18:08.778-08:00Talend Project - Child jobs, Functions and Variables<div class="separator" style="clear: both; text-align: left;">
This concluding part of Talend project video explores creating sub/child jobs, how to pass data between the two and different strategies to share the data.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioIlIqDGwD0uMbhnXncpQerxI4Tjbb9NQbI4ykiDrZiYcmPUu2_GFS5McgpYhI3m1GizCIVCNdiSeAm3In8A7W25_7cbZwZ7UGMd-zGwWnWsu9qWAADpc1xQQD0Qlco8TxTijRIA/s1600/parent.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="187" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioIlIqDGwD0uMbhnXncpQerxI4Tjbb9NQbI4ykiDrZiYcmPUu2_GFS5McgpYhI3m1GizCIVCNdiSeAm3In8A7W25_7cbZwZ7UGMd-zGwWnWsu9qWAADpc1xQQD0Qlco8TxTijRIA/s400/parent.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Code in subroutines</div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">// Code from https://www.youtube.com/watch?v=a7-HUU4js9E</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">package routines;</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">public class formatEmails {</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> public static String formatEmail(char oldRating, char newRating, String typeOfRating) {</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> String fEmailText="";</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>if (newRating>oldRating) {</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>fEmailText=fEmailText+"<span style="color: red;">"+typeOfRating +" rating reduced from " +Character.toString(oldRating) +" to "+Character.toString(newRating) +"</span><br />" ;</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> }</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>if (newRating<oldrating font=""></oldrating></span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>fEmailText=fEmailText+"<span style="color: green;">"+typeOfRating +" rating improved from " +Character.toString(oldRating) +" to "+Character.toString(newRating) +"</span><br />" ;</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> }</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>return fEmailText;</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;"> }</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: "courier new" , "courier" , monospace;">}</span></div>
<div style="text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
It also explores subroutines/functions that helps you reduce the code and modularizes it. </div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg58nbYnNMtSi3_q0dP_IN56JibaDitPtYGBlRJgnd1MNKaYcauxoVG04fIcseDv25pdRw8fjATSZjaD7PevjtNo1zSpWhEtNO0qwLUk-2SZ5GxDavxjeU33AHystQ0qO2Wz8rxug/s1600/sub.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="182" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg58nbYnNMtSi3_q0dP_IN56JibaDitPtYGBlRJgnd1MNKaYcauxoVG04fIcseDv25pdRw8fjATSZjaD7PevjtNo1zSpWhEtNO0qwLUk-2SZ5GxDavxjeU33AHystQ0qO2Wz8rxug/s400/sub.png" width="400" /></a></div>
<br />
Check out the detailed video - <a href="https://www.youtube.com/watch?v=a7-HUU4js9E">https://www.youtube.com/watch?v=a7-HUU4js9E</a><br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/a7-HUU4js9E/0.jpg" frameborder="0" height="296" src="https://www.youtube.com/embed/a7-HUU4js9E?feature=player_embedded" width="520"></iframe></div>
<br />
tjavarow component code<br />
<span style="font-family: "courier new" , "courier" , monospace;">/* -- Code from https://www.youtube.com/watch?v=a7-HUU4js9E ---- */ </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">String wholepage; </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">String ratings;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">wholepage=input_row.document.toString(); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">int pos=wholepage.indexOf("composite_val"); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">ratings=wholepage.substring(pos,pos+250).replaceAll("[\\[\\]\"]", "").replaceAll(" \n", " ").replaceAll(" composite_val_vgm",""); <span class="Apple-tab-span" style="white-space: pre;"> </span> String allratingsonly="";</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">String[] splitratings = ratings.split("composite_val>"); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">int i=0;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">context.EmailText=context.EmailText+"<br /><br /><b><span style="color: black;">Ratings for : " +context.stock +"<b></b></span><br />" ;</b></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> for (String eachratingrow : splitratings) </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> { </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>if (eachratingrow.length()>0) </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> { </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span></span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>if (i==0){ </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> output_row.z_growth_rating=Character.toString(eachratingrow.charAt(0)); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> context.EmailText=context.EmailText+formatEmails.formatEmail(input_row.growth_rating.charAt(0), eachratingrow.charAt(0), "Growth");</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>} </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>if (i==1)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> output_row.z_momentum_rating=Character.toString(eachratingrow.charAt(0)); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> context.EmailText=context.EmailText+formatEmails.formatEmail(input_row.momentum_rating.charAt(0), eachratingrow.charAt(0), "Momentum"); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> } </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>if (i==2)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>{ </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>output_row.z_value_rating=Character.toString(eachratingrow.charAt(0)); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>context.EmailText=context.EmailText+formatEmails.formatEmail(input_row.value_rating.charAt(0), eachratingrow.charAt(0), "Value"); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>} </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>if (i==3)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span>{ </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> output_row.z_vgm_rating=Character.toString(eachratingrow.charAt(0)); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> context.EmailText=context.EmailText+formatEmails.formatEmail(input_row.vgm_rating.charAt(0), eachratingrow.charAt(0), "VGM"); </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> } </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> <span class="Apple-tab-span" style="white-space: pre;"> </span> i++;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> }</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> } </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">output_row.EmailText=context.EmailText;</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> /* - End of Code from https://www.youtube.com/watch?v=a7-HUU4js9E --*/</span><br />
<br />sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com1tag:blogger.com,1999:blog-7798377.post-89696613728942001912017-01-07T09:18:00.001-08:002017-02-05T04:47:13.939-08:00Talend and Cassandra - Storing data and retrievingThis video explores saving data in Cassandra NoSql database. This video is a continuation of previous one. Apart from saving the data, it also discusses about quick tips on strategies for creating, storing and retrieving data in Cassandra.<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/KMCzJkgHv5s/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/KMCzJkgHv5s?feature=player_embedded" width="480"></iframe></div>
<br />
<h3>
Code used in the video</h3>
<span style="font-family: "trebuchet ms" , sans-serif;">Select * from ss.stockratings where stock='TSLA' order by ratingtime desc limit 1;</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">Insert into ss.ratings(stock ,ratingtime,id,growth_rating,momentum_rating, <span class="Apple-tab-span" style="white-space: pre;"> </span>value_rating,vgm_rating) values ('TSLA',dateof(now()),now(),'A','D','A','F');</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"><br /></span>
<span style="font-family: "trebuchet ms" , sans-serif;">CREATE TABLE ss.stockratings (</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"> id uuid,</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"> stock text,</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"> ratingtime timestamp,</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"> growth_rating text,</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"> momentum_rating text,</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"> value_rating text,</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"> vgm_rating text,</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;"> PRIMARY KEY ( stock, ratingtime,id)</span><br />
<span style="font-family: "trebuchet ms" , sans-serif;">);</span>sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-1596523739051916782017-01-05T04:26:00.000-08:002017-01-17T18:34:22.204-08:00Talend Project - Send mail (tSendMail component)<a href="http://www.talend.com/" target="_blank">Talend</a>'s tSendMail component can be used to send HTML formatted emails. this Video demonstrates framing a HTML formatted text using the project we were working on.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzXBVZo4LnRRQSERWHEnsH29Cxu94bbPw_sg951ZTpfyoZlhgng0wAb9ahRi4axbBwoKIjKB1GKbS-nhrjmxO584hEGAe4-GvjeyvjK4Y1gewryN579Dx4pPT4l69V0CvTgvLgBg/s1600/sendmail.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="258" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzXBVZo4LnRRQSERWHEnsH29Cxu94bbPw_sg951ZTpfyoZlhgng0wAb9ahRi4axbBwoKIjKB1GKbS-nhrjmxO584hEGAe4-GvjeyvjK4Y1gewryN579Dx4pPT4l69V0CvTgvLgBg/s400/sendmail.png" width="400" /></a></div>
<br />
This is the code that we used in tJavaRow component to extract the rating and create an email body that lists and changes compared to previous rating. Please follow the video get a complete understanding.<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "courier new" , "courier" , monospace;">/* -- Code from https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ ---- */ </span><span style="font-family: "courier new" , "courier" , monospace;">String wholepage; </span><span style="font-family: "courier new" , "courier" , monospace;">String ratings; </span><span style="font-family: "courier new" , "courier" , monospace;">wholepage=input_row.document.toString(); </span><span style="font-family: "courier new" , "courier" , monospace;">int pos=wholepage.indexOf("composite_val"); </span><span style="font-family: "courier new" , "courier" , monospace;">ratings=wholepage.substring(pos,pos+250).replaceAll("[\\[\\]\"]", "").replaceAll(" \n", " ").replaceAll(" composite_val_vgm",""); </span><span style="font-family: "courier new" , "courier" , monospace;">//output_row.document = ratings; </span><span style="font-family: "courier new" , "courier" , monospace;">String allratingsonly=""; </span><span style="font-family: "courier new" , "courier" , monospace;">String[] splitratings = ratings.split("composite_val>"); </span><span style="font-family: "courier new" , "courier" , monospace;">int i=0;</span><span style="font-family: "courier new" , "courier" , monospace;">String EmailText="Ratings for : " +context.stock +"</span><span style="font-family: "courier new" , "courier" , monospace;">" ;</span><span style="font-family: "courier new" , "courier" , monospace;">for (String eachratingrow : splitratings) </span><span style="font-family: "courier new" , "courier" , monospace;">{ </span><span style="font-family: "courier new" , "courier" , monospace;"> if (eachratingrow.length()>0)</span><span style="font-family: "courier new" , "courier" , monospace;"> { allratingsonly=allratingsonly+";"+ eachratingrow.charAt(0)+""; //allratingsonly=allratingsonly+eachratingrow+"**;"; </span><span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;"> if (i==0){</span><span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>output_row.z_growth_rating=Character.toString(eachratingrow.charAt(0)); <span class="Apple-tab-span" style="white-space: pre;"> </span> </span><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>if ( eachratingrow.charAt(0)>input_row.growth_rating.charAt(0))</span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> { </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>EmailText=EmailText+"<span style="color: red;">Growth rating reduced from " +input_row.growth_rating+ " To "+ <span class="Apple-tab-span" style="white-space: pre;"> </span>eachratingrow.charAt(0)+"<br />"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="color: red;">"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> }</span><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;"> if ( eachratingrow.charAt(0)</span><span style="font-family: "courier new" , "courier" , monospace;"> { </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>EmailText=EmailText+"<span style="color: green;">Growth rating improved from " +input_row.growth_rating +" To "+ <span class="Apple-tab-span" style="white-space: pre;"> </span>eachratingrow.charAt(0)+"<br />"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="color: green;">"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"> }</span><span style="font-family: "courier new" , "courier" , monospace;"> }</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;"> if (i==1){</span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>output_row.z_momentum_rating=Character.toString(eachratingrow.charAt(0));</span><span class="Apple-tab-span" style="font-family: "courier new" , "courier" , monospace; white-space: pre;"> </span><span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>if ( eachratingrow.charAt(0)>input_row.momentum_rating.charAt(0))</span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> { </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>EmailText=EmailText+"<span style="color: red;">momentum rating reduced from " +input_row.momentum_rating+ " To "+ <span class="Apple-tab-span" style="white-space: pre;"> </span>eachratingrow.charAt(0)+"<br />"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="color: red;">"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> }</span><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;"> if ( eachratingrow.charAt(0)</span><span style="font-family: "courier new" , "courier" , monospace;"> { </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>EmailText=EmailText+"<span style="color: green;">momentum rating improved from " +input_row.momentum_rating +" To "+ <span class="Apple-tab-span" style="white-space: pre;"> </span>eachratingrow.charAt(0)+"<br />"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="color: green;">"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"> }</span><span style="font-family: "courier new" , "courier" , monospace;"> }</span><span style="font-family: "courier new" , "courier" , monospace;"> if (i==2){</span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>output_row.z_value_rating=Character.toString(eachratingrow.charAt(0));</span><span class="Apple-tab-span" style="font-family: "courier new" , "courier" , monospace; white-space: pre;"> </span><span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>if ( eachratingrow.charAt(0)>input_row.value_rating.charAt(0))</span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> { </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>EmailText=EmailText+"<span style="color: red;">value rating reduced from " +input_row.value_rating+ " To "+ <span class="Apple-tab-span" style="white-space: pre;"> </span>eachratingrow.charAt(0)+"<br />"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="color: red;">"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> }</span><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;"> if ( eachratingrow.charAt(0)</span><span style="font-family: "courier new" , "courier" , monospace;"> { </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>EmailText=EmailText+"<span style="color: green;">value rating improved from " +input_row.value_rating +" To "+ <span class="Apple-tab-span" style="white-space: pre;"> </span>eachratingrow.charAt(0)+"<br />"; <span class="Apple-tab-span" style="white-space: pre;"> </span></span></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="color: green;">"; <span class="Apple-tab-span" style="white-space: pre;"> </span></span></span><span style="font-family: "courier new" , "courier" , monospace;"> }</span><span style="font-family: "courier new" , "courier" , monospace;"> }</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;"> if (i==3){</span><span style="font-family: "courier new" , "courier" , monospace;"> </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>output_row.z_vgm_rating=Character.toString(eachratingrow.charAt(0));</span><span class="Apple-tab-span" style="font-family: "courier new" , "courier" , monospace; white-space: pre;"> </span><span style="font-family: "courier new" , "courier" , monospace;"> <span class="Apple-tab-span" style="white-space: pre;"> </span>if ( eachratingrow.charAt(0)>input_row.vgm_rating.charAt(0))</span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> { </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>EmailText=EmailText+"<span style="color: red;">vgm rating reduced from " +input_row.vgm_rating+ " To "+ <span class="Apple-tab-span" style="white-space: pre;"> </span>eachratingrow.charAt(0)+"<br />"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="color: red;">"; </span></span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span> }</span><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;"> if ( eachratingrow.charAt(0)</span><span style="font-family: "courier new" , "courier" , monospace;"> { </span><span style="font-family: "courier new" , "courier" , monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>EmailText=EmailText+"<span style="color: green;">vgm rating improved from " +input_row.vgm_rating +" To "+ <span class="Apple-tab-span" style="white-space: pre;"> </span>eachratingrow.charAt(0)+"<br />"; <span class="Apple-tab-span" style="white-space: pre;"> </span></span></span><span style="font-family: "courier new" , "courier" , monospace;"><span style="color: green;">"; <span class="Apple-tab-span" style="white-space: pre;"> </span></span></span><span style="font-family: "courier new" , "courier" , monospace;"> }</span><span style="font-family: "courier new" , "courier" , monospace;"> }</span><span style="font-family: "courier new" , "courier" , monospace;"> i++;</span><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;"> } </span><span style="font-family: "courier new" , "courier" , monospace;">} </span><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><span style="font-family: "courier new" , "courier" , monospace;">output_row.EmailText=EmailText;</span><span style="font-family: "courier new" , "courier" , monospace;">/* - End of Code from https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ --*/</span><br />
<br />
<br /></blockquote>
<div>
<div>
<div>
<div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/kd_oibgk8qU/0.jpg" frameborder="0" height="333" src="https://www.youtube.com/embed/kd_oibgk8qU?feature=player_embedded" width="580"></iframe></div>
<br /></div>
If you haven't visited this project from the beginning, Here is the first post about it in this blog - <a href="http://sanjaykattimani.blogspot.com/2016/12/talend-project-to-parse-webpage-zackscom.html">http://sanjaykattimani.blogspot.com/2016/12/talend-project-to-parse-webpage-zackscom.html</a>
</div>
</div>
</div>
</div>
sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-4808575754828877242016-12-17T19:16:00.002-08:002017-01-17T18:33:27.426-08:00Talend project to parse a webpage (Zacks.com)Created another interesting Talend project over the weekend. This <a href="https://www.youtube.com/watch?v=Zit7Pu_az8E" target="_blank">Talend job parses zacks.com webpage to extract zacks scores</a> and then convert them to rows that can be used in other components. tHTMLParse compent use to parse the website is available in Talend's exchange (market place) for free. String manipulation consumed majority of my time on this project. I intend to extend this project in future<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/Zit7Pu_az8E/0.jpg" frameborder="0" height="320" src="https://www.youtube.com/embed/Zit7Pu_az8E?feature=player_embedded" width="580"></iframe></div>
<br />
Here is the code that goes in tJavaRow component that extracts only ratings out of the whole page and returns a string of ratings separated by semicolon.<br />
<span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">/* -- Code from https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ ---- */ </span></span><br />
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">String wholepage; </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">String ratings; </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">wholepage=input_row.document.toString(); </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">int pos=wholepage.indexOf("composite_val"); </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">ratings=wholepage.substring(pos,pos+250).replaceAll("[\\[\\]\"]", "").replaceAll(" \n", " ").replaceAll(" composite_val_vgm",""); </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">//output_row.document = ratings; </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">String allratingsonly=""; </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">String[] splitratings = ratings.split("composite_val>"); </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">for (String eachratingrow : splitratings) </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">{ </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> if (eachratingrow.length()>0)</span></div>
<div>
<span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;"> { </span><span style="font-family: "courier new" , "courier" , monospace;">allratingsonly=allratingsonly+";</span><span style="font-family: "courier new" , "courier" , monospace;">"+ eachratingrow.charAt(0)+""; //allratingsonly=allratingsonly+eachratingrow+"**;"; </span></span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;"> } </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">} </span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">output_row.document=allratingsonly;</span><br />
<span style="font-family: "courier new" , "courier" , monospace; font-size: x-small;">/* - End of Code from https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ --*/</span></div>
sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0Boston, MA, USA42.3600825 -71.0588801000000141.984348999999995 -71.704327100000015 42.735816 -70.4134331tag:blogger.com,1999:blog-7798377.post-83185774495357086842016-11-28T21:03:00.002-08:002017-01-22T17:56:17.970-08:00Creating MEAN stack application using Bitnami launchpad in AWSMy earlier Harvard course on Bigdata Analytics had given me enough exposure on <a href="http://aws.amazon.com/" target="_blank">AWS </a>(Amazon Web Services) and professional and personal curiosity on emerging technologies taught me enough about MEAN stack. MEAN stack is a full stack development tool kit that includes javascript based techologies MongoDB, Express.js, Angular.js and Node.js. <br />
<br />
I wanted to take this learning to next level by hosting a <a href="https://www.youtube.com/watch?v=j1--f1yMSB0" target="_blank">MEAN stack application on AWS instance</a>. While self exploring some intricate details of configuring and using AWS, much of the documentation was scattered to address individual needs and I had to connect the dots to meet my needs. Some of the documentation was old and some of the screenshots were obsolete. I decided to make a video along the way so that people who look for such information also get to learn. So here is the video of <a href="http://creating%20mean%20stack%20application%20using%20bitnami%20launchpad%20in%20aws/" target="_blank">Creating MEAN stack application using Bitnami launchpad in AWS</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/-J_mpTYwrEs/0.jpg" frameborder="0" height="350" src="https://www.youtube.com/embed/-J_mpTYwrEs?feature=player_embedded" width="580"></iframe><br />
<br />
I intend to do further videos explaining about other parts such as how to connect to the newly created VM, using Rockmongo, etc<br />
<br />
Feel free to ask any related questions by leaving comments below this post or comments in <a href="https://www.youtube.com/watch?v=j1--f1yMSB0" target="_blank">youtube video page</a>.sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-7389231943599852122015-02-07T06:07:00.000-08:002017-04-30T06:21:22.416-07:00Big Data Analytics at Harvard Extension School<br />
Quest for upgrades in technologies and skills are indispensable traits in software field. Although it was my impulsive decision to join this course, fortunately everything worked out very well to start the studies at <a href="http://www.extension.harvard.edu/" target="_blank">Harvard extension school </a>in Boston, Massachusetts. <a href="http://www.extension.harvard.edu/courses/big-data-analytics" target="_blank">Big data analytics</a> subject that a friend had enrolled intrigued me as I have been thinking of learning about this subject for a long time. Big data also aligns very well with my existing knowledge of over 15 years of Business Intelligence, <a href="https://www.youtube.com/playlist?list=PLUxVA4dlJgcy8-WB0VM47phVE4Mr4uL9F" target="_blank">ETL</a> and databases related experience.<br />
<br />
I have been working on Healthcase analytics for over 3 years and it seemed like a best fit course coupled with the thirst to learn were all solved by this course.<br />
Here is my project that i presented at the end of the course. It was one of the very few projects selected to be presented at Harvard to all our class.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://www.youtube.com/embed/H3OmS6he8lg?feature=player_embedded" frameborder="0" height="350" src="https://www.youtube.com/embed/H3OmS6he8lg?feature=player_embedded" width="580"></iframe></div>
<br />
<br />
This course instantly prepared me for the next logical shift in my career and as a result of all the curiosity, I ended up creating a <a href="https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ" target="_blank">BigData Technologies Youtube channel</a> - <a href="https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ">https://www.youtube.com/channel/UCT3bqK2QL93j-IFYFYbvjWQ</a> to share my technical explorations.<br />
<div>
<br /></div>
<br />sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com01 Story Street, Cambridge, MA 02138, USA42.3741318 -71.12182969999997818.5203023 -112.43042369999998 66.2279613 -29.813235699999979tag:blogger.com,1999:blog-7798377.post-64551978903241410562014-11-20T06:29:00.000-08:002017-04-30T09:56:18.108-07:00Wish list for Software Systems integration in every countryRecently I visited RMV in Watertown, MA for getting my driving license updated. They all did their job as best as possible. Being an Integration Consultant, every time I visit any government agency like this, I keep thinking about this.<br />
<br />
They all have every bit of information already about me in their (at least one of their) systems. WHY DO THEY ASK ME TO FILL THESE SAME QUESTIONS IN EVERY FORM?<br />
They waste<br />
<ul>
<li>My time in finding the information, filling, correcting, printing, taking time out of my office hours </li>
<li>Govt agency's time in validating every details of hand written (with human errors) applications, providing facilities for.</li>
<li>National resources - Infrastructure for buildings etc.</li>
</ul>
<div>
My wish list for future (you would think that this is right out of a sci-fi movie) is</div>
<div>
<ol>
<li>When you walk in to airports, No visa, passport, ticket or any other document to be presented instead scan my iris, palm or an embedded device. Do not ask me the purpose of visit, you must already be knowing it with verified documents (air ticket, valid visa and passport)</li>
<li>Our medical records</li>
<ol>
<li>They should be shared by all physicians/clinics.</li>
<li>I should not be filling all those forms again with detailed clinical history at every clinic.</li>
</ol>
<li>If I need to transfer my driving license/car registration etc, System should already know that i moved from Texas to Massachusetts and should present me a confirm button with every update its going to do namely,</li>
<ol>
<li>Insurance update</li>
<li>New driving license issued by the state</li>
<li>Car and other vehicles registration transferred </li>
</ol>
</ol>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://rishab.files.wordpress.com/2013/03/plpc176.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="159" src="https://rishab.files.wordpress.com/2013/03/plpc176.png" width="320" /></a></div>
<div>
<br /></div>
<div>
There are numerous use cases like these where we can automate<br />
Although these might sound like next to impossible task given the different agencies involved that govern the data, but we already have most of the information stored in multiple systems and its not an impossible task to integrate these. </div>
</div>
<div>
</div>
<div>
<br />
<br /></div>
sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-53518546165804959792014-03-20T16:19:00.000-07:002014-04-29T17:43:26.396-07:00SoapUI - A powerful web services testing tool<div style="background-color: white;">
<span style="color: #222222; font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">Recently I happened to Review/Test a few web services implemented for one of my client. </span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXU0ZG1rmSmGA_49agMkol1PbmD2iwFQvza7qdPUb8JDMidpmeOdexZzz6zPDeoZzw5iMBaAw2bWsO3p-Eugmcve1MOeNhI762EdoCf8HKUOHzRUh_Fj8D-blQR6J3FGMhj13_7A/s1600/soappro.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="SoapUI Pro" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXU0ZG1rmSmGA_49agMkol1PbmD2iwFQvza7qdPUb8JDMidpmeOdexZzz6zPDeoZzw5iMBaAw2bWsO3p-Eugmcve1MOeNhI762EdoCf8HKUOHzRUh_Fj8D-blQR6J3FGMhj13_7A/s1600/soappro.jpg" title="SoapUI Pro" /></a></div>
<span style="color: #222222; font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><br /></span>
<span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">SoapUI is the defacto standard for validating webservices. Apart from technical deliverables, I also got to evaluate the product and compare it with free version. I downloaded free version and Pro (14 days evaluation) and was pleasantly surprised at the advancements and ease of use. I developed webservices a few years ago and hardly any useful tool was there to test it. </span><br />
<div>
<span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;"><br /></span></div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCgNWiJWAPeEV7tK1TwUKepgIqWtrFXDTxMHg_Dk6lFXma2gIOchuPAr17NR8vM5PmiiLnN1aeTUv23rA-4Z4czyvyGAp3Husf2ggFFfct_lolo9gN5H0DeU0dn-HvcYOJfZx5pw/s1600/soapui.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="SoapUI Pro interface" border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCgNWiJWAPeEV7tK1TwUKepgIqWtrFXDTxMHg_Dk6lFXma2gIOchuPAr17NR8vM5PmiiLnN1aeTUv23rA-4Z4czyvyGAp3Husf2ggFFfct_lolo9gN5H0DeU0dn-HvcYOJfZx5pw/s1600/soapui.jpg" height="192" title="SoapUI Pro interface" width="320" /></a><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"> </span></span><br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;"><br /></span>
<span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Some of the notable features I liked and used are</span><br />
<br />
<ul>
<li><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Dynamic Duo/The Editors - You can visually enter data rather than searching though XML and entering the data.</span><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;"> I used this extensively and is a very neat feature. </span></li>
<li><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Point and Click Testing - Almost ready to test the webservices just with few clicks. You can click on the web service response xml contents and right click and few more clicks and SoapUI writes the xpath qeury and associated test case. Its that simple. </span></li>
<li><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Test Debugging - Helps creating test flows, other testing related constructs like conditional execution, looping etc. I mostly utilized connecting to DB (data source) and groovy script.</span></li>
<li><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Data Sources - Provides options to connect to most popular data sources. You can write queries/get the data from these sources and then use use the data as part of inputs to your webservice calls or compare the data to validate the webservice response.</span></li>
<li><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Multi-Environment Support - If you have multiple environments on which you need to do same webservice testing, this is a major timesaver. I created my own config file and imported those settings as project's properties. Client code, end points. connection string, user name and such commonly used strings were stored and the imported properties could be used in test cases as variables with a few clicks(Again I did not type any of these).</span></li>
<li><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Reporting - Provides nice reporting of all test suites/cases executed with their status. You can also customize the template of reporting. I mostly created HTML reports as they were very easy to use and store but PDF, CSV and other formats are also available. </span></li>
<li><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">The SQL builder - Sql builder allows you to visually choose the tables and build query, our huge database with few thousand schemas was too much for SoapUI . I preferred using my ready to use queries and it worked flawlessly.</span></li>
<li><span style="color: #222222; font-family: 'Helvetica Neue', Arial, Helvetica, sans-serif;">Stop the Hackers - XML Bomb, sql injection, malformed XML and plenty of other ready to use test cases are available under security category - Just few clicks away. </span></li>
</ul>
<br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;">SoapUI had plenty of other features but these were the features that I ended up using and was pleased with the self explanatory and user friendly interface.</span><br /><span style="color: #222222;">You can take a look at their <a href="http://www.soapui.org/Go-Pro/12-reasons-to-go-pro.html">SoapUI : 12 reasons to go pro</a> page for more details.</span></span><br />
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;"><br /></span></span>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><span style="color: #222222;">SoapUI also has a free version. Free version is very limited but lets you create your test cases with basic features.</span></span></div>
sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com6tag:blogger.com,1999:blog-7798377.post-37734392184649389142013-05-18T23:31:00.001-07:002013-05-19T06:22:27.142-07:00Adobe LightRoom<br />
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">Tried Adobe LightRoom software today on some of my not so beautiful looking Raw images (of course bluebonnet flowers and kids are always beautiful)</span></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;">What a difference. </span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguYSRd3xZCb5w0sfkFarXsYPHDuPWn7-eHvkNEkuntzteakg-xidyz8mNwVzfXN8tT5VygXgZLXWLXV1Hs1Xqos5d_uu0flk4oeBgleTvdHRMgVCM67GWAlxfI-Dm3CoyGDyFQ/s1600/light+room.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"><img border="0" height="190" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguYSRd3xZCb5w0sfkFarXsYPHDuPWn7-eHvkNEkuntzteakg-xidyz8mNwVzfXN8tT5VygXgZLXWLXV1Hs1Xqos5d_uu0flk4oeBgleTvdHRMgVCM67GWAlxfI-Dm3CoyGDyFQ/s400/light+room.png" width="400" /></span></a></div>
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif;"> <span style="text-align: center;">I always wondered at those blank skies in my images and now i know how to deal with them. Pictures from Heavenly Bokeh group on Facebook and references from a few friends there inspired me to install and try out these effects. Burning midnight oil was well worth it. </span></span><br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Helvetica Neue, Arial, Helvetica, sans-serif; margin-left: 1em; margin-right: 1em;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLGLKTWThPtyNSkZYZNM0RxgWp5QCA0KME9fgaJbiBljYICtrA33VUGQ3a_4tnFM6JaCKiNm_4nQEXtId2ePFn0HloOWjb6rWaDjkPyQX7gpGH7T4fydoAGcMyDUQhwBf38-Nv/s1600/IMG_6148.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLGLKTWThPtyNSkZYZNM0RxgWp5QCA0KME9fgaJbiBljYICtrA33VUGQ3a_4tnFM6JaCKiNm_4nQEXtId2ePFn0HloOWjb6rWaDjkPyQX7gpGH7T4fydoAGcMyDUQhwBf38-Nv/s400/IMG_6148.jpg" width="400" /></a></span></div>
<div>
<br /></div>
sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-17779170557926921152012-03-06T07:50:00.000-08:002017-01-22T18:00:28.921-08:00Youtube is scanning your videos<div style="border-bottom: medium none; border-left: medium none; border-right: medium none; border-top: medium none;">
<span style="font-family: "georgia" , "times new roman" , serif;">Yesterday I uploaded a video on Youtube that I created with photos and used some background music that i downloaded from net. In an hour, Youtube sent the following email to me.</span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizj_Y_DTQkr7eLtVMKDAbnfhxadnl4wOFNWr2Dvdf57IZIfw3YogbU13MvgOnapFZY-7Lp9AS5VEEand6prbK1ppGYATay06hJFCT09PmqByTDbPA_AcIYFA8TV7eg6peTEb1qgA/s1600/YouTube-and-SEO.jpg" imageanchor="1" style="clear: right; cssfloat: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="141" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizj_Y_DTQkr7eLtVMKDAbnfhxadnl4wOFNWr2Dvdf57IZIfw3YogbU13MvgOnapFZY-7Lp9AS5VEEand6prbK1ppGYATay06hJFCT09PmqByTDbPA_AcIYFA8TV7eg6peTEb1qgA/s200/YouTube-and-SEO.jpg" uda="true" width="200" /></a></div>
<div style="border-bottom: medium none; border-left: medium none; border-right: medium none; border-top: medium none;">
<br /></div>
<div style="border-bottom: medium none; border-left: medium none; border-right: medium none; border-top: medium none;">
<br /></div>
<div style="border-bottom: medium none; border-left: medium none; border-right: medium none; border-top: medium none;">
<span style="font-family: "georgia" , "times new roman" , serif;">Dear sanjaykattimani, </span></div>
<div style="border-bottom: medium none; border-left: medium none; border-right: medium none; border-top: medium none;">
<span style="font-family: "georgia" , "times new roman" , serif;">Your video, RishabHomeVideo.m2ts, may have content that is owned or licensed by Yash Raj Films Pvt. Ltd.. </span><span style="font-family: "georgia" , "times new roman" , serif;">No action is required on your part; however, if you are interested in learning how this affects your video, please visit the Content ID Matches section of your account for more information. </span></div>
<div style="border-bottom: medium none; border-left: medium none; border-right: medium none; border-top: medium none; text-align: left;">
<span style="font-family: "georgia" , "times new roman" , serif;">Sincerely,</span></div>
<div style="text-align: left;">
<br /></div>
<div style="border-bottom: medium none; border-left: medium none; border-right: medium none; border-top: medium none; text-align: left;">
<span style="font-family: "georgia";">- The YouTube Team</span></div>
<div style="text-align: center;">
<span style="color: #9fc5e8; font-family: "georgia" , "times new roman" , serif;"> </span><span style="color: #9fc5e8; font-family: "georgia" , "times new roman" , serif;">© 2012 YouTube, LLC<br />901 Cherry Ave, San Bruno, CA 94066 </span></div>
<div style="text-align: center;">
<br /></div>
<span style="font-family: "georgia" , "times new roman" , serif;"><span style="font-family: "georgia" , "times new roman" , serif;"><strong>So whats Google/Youtube up to?</strong></span><br />
<span style="font-family: "georgia" , "times new roman" , serif;">Youtube is scanning all the videos that you are getting uploaded and scanning for music that matches their video/audio library. In case it matches, sends out the above warning message. This is a smart way to reduce/fight copyright infringements.</span><br />
May be this was part of Googles privacy policy changes. I didn't read them. Did you?<br />
<br />
<span style="font-family: "georgia" , "times new roman" , serif;">I feel this is the best for copyright owners as they are suppose to get their share. At the same time if they start going after users like us to collect royalty, we would rather drop those songs and use some general public licensed music. Or better, I may end up creating my own music?? :)</span><br />
</span><br />
<span style="font-family: "georgia" , "times new roman" , serif;"><span style="font-family: "georgia" , "times new roman" , serif;"><br /></span></span>
<span style="font-family: "georgia" , "times new roman" , serif;"><span style="font-family: "georgia" , "times new roman" , serif;">Update 1/22/2017 - Noticed that YouTube still lets you continue to upload your videos, It certainly does not let you monetize (Earn on the ads) your videos. Its not hard to find royalty free music for most of your compositions though.</span></span><br />
<div style="text-align: center;">
<span style="font-family: "georgia" , "times new roman" , serif;"></span></div>
<span style="font-family: "georgia" , "times new roman" , serif;">
</span>sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com11tag:blogger.com,1999:blog-7798377.post-39433741623354068422011-05-24T20:30:00.000-07:002011-06-05T10:48:01.715-07:00Spamming gets innovative.<span style="font-family: Arial, Helvetica, sans-serif;">Spammers Hakckers keep finding new and innovative ways to reach out to the masses. Since facebook has been immensely popular social network, it becomes vert attractive media for spammers. It also seems to be very easy to convince users to click a few buttons and steal all your data. They also post links on your friend's walls to attract more visitors.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span><br />
<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQrasWw4aYtIBLcu5vRCV2ODem1u0hY-iTzP4V76Fx-iLrKtJaMU-yG7r98Ax8ppM0qPP1wFoblao9D9mD2e5A1HzdyD-rr89tdeZpNpmucx_nQs5a4V6BG4beE-iLiVUa5_1eUQ/s1600/spam.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-family: Arial, Helvetica, sans-serif;"><img border="0" height="261" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQrasWw4aYtIBLcu5vRCV2ODem1u0hY-iTzP4V76Fx-iLrKtJaMU-yG7r98Ax8ppM0qPP1wFoblao9D9mD2e5A1HzdyD-rr89tdeZpNpmucx_nQs5a4V6BG4beE-iLiVUa5_1eUQ/s400/spam.png" width="400" /></span></a></div><span style="font-family: Arial, Helvetica, sans-serif;"><br />
</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Found a couple of my innocent friends posting these materials again and again. </span><br />
<strong><span style="font-family: Arial, Helvetica, sans-serif;">How do you safeguard youself?</span></strong><br />
<ul><li><span style="font-family: Arial, Helvetica, sans-serif;">Dont click on any link that you do not know or is suspicious.</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">If any application asks you to approve/join/like do not do it unless you know the website/link</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">They will have very attractive images/text - Eg. Osama bin Laden's uncensored pictures. If its too good to be true, it may not be safe link.</span></li>
</ul><br />
<strong><span style="font-family: Arial, Helvetica, sans-serif;">What do they get by spreading these kinds of links?</span></strong><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Any of the following</span><br />
<ul><li><span style="font-family: Arial, Helvetica, sans-serif;">Viral marketing - Website may be trying to make some product/website popular</span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Stealing your details - They may be collecting your name, geographic location phone etc to sell it further. </span></li>
<li><span style="font-family: Arial, Helvetica, sans-serif;">Stealing your id and passwords .</span></li>
</ul><span style="font-family: Arial, Helvetica, sans-serif;">So please safeguard our accounts and share the good messages/links/videos with your friends. Do waste your and your friend's time and energy by clicking the wrong links. </span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Happy socializing.</span>sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com8tag:blogger.com,1999:blog-7798377.post-65886965773416817982011-05-19T19:36:00.000-07:002011-05-19T19:36:01.677-07:00QR CodesDo you know what is QR Code?<br />
A QR code (short for Quick Response) is a specific matrix barcode (or two-dimensional code), readable by dedicated QR barcode readers and camera phones. The code consists of black modules arranged in a square pattern on a white background. The information encoded can be text, URL or other data.<div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDGyw7ad3HAPYrczlI2Xwc6QOO1snY_iFaQYe_5KUtf5j6U9kjRISUZGdGZv2oTNOcMRsKw0R65zb_wj69bkxQjSmsHL-jNuylHy2AJwwaDpQt7nIm5EJ2qbi1t1KJuTtTg_hFrQ/s1600/chart+%25281%2529.png" imageanchor="1" style="clear:right; float:right; margin-left:1em; margin-bottom:1em"><img border="0" height="230" width="230" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDGyw7ad3HAPYrczlI2Xwc6QOO1snY_iFaQYe_5KUtf5j6U9kjRISUZGdGZv2oTNOcMRsKw0R65zb_wj69bkxQjSmsHL-jNuylHy2AJwwaDpQt7nIm5EJ2qbi1t1KJuTtTg_hFrQ/s400/chart+%25281%2529.png" /></a></div><br />
<br />
Was checking a barcode reader and found that app that i was testing can read QR codes so this was it.<br />
Here is a QR code of my <a href="http://www.kannada-greetings.com">Kannada greetings</a> site.<br />
<br />
These are easily readable by most smartphones with camera by installing small app.sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com2tag:blogger.com,1999:blog-7798377.post-65825897165197743512011-03-29T03:47:00.000-07:002016-11-29T03:48:20.755-08:00Garmin nüvi 1450 5-Inch Portable GPS Navigator<table border="0"><tbody>
<tr><td>My previous Tom Tom XL got banged inside car many times as the suction holder gave up because prolonged direct sunlight/heat. It had lost the speaker for some time, and recently after it was handled by a friend, it died completely. Time to buy a new GPS. Its a very valuable tool if you travel even little bit and not sure about your local roads. It saves lot of time, gas and most importantly guides you back on the correct road when you go in a wrong route.<br />
Gramin is one the most popular names in navigation today<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrYVKrAFq74WAPp2byOWLKwsmDKAQa1NGcU17Hu3VV6slXxnzCNhFvQlfbrDW3lGvenmgtBPg2yOtIP1MiC9CrIsSaGFpSlmHL6VP_Jtia9lkVsP_kYSphut90oYTYPyFAqNcSXA/s1600/Garmin-Nuvi-1490T-R-Nuvi-1490_250x220.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrYVKrAFq74WAPp2byOWLKwsmDKAQa1NGcU17Hu3VV6slXxnzCNhFvQlfbrDW3lGvenmgtBPg2yOtIP1MiC9CrIsSaGFpSlmHL6VP_Jtia9lkVsP_kYSphut90oYTYPyFAqNcSXA/s1600/Garmin-Nuvi-1490T-R-Nuvi-1490_250x220.jpg" /></a></div>
<br />
I have found this invaluable tool to navigate specially the urban part of western countries.<br />
<h3>
Update: </h3>
5 years writing this article, Both cars i bought have inbuilt GPS so that i do not have to scramble to connect gps or hook it to the windshield on need basis. In future they will all be tightly integrated in in cars rather than being optional. I already see this in cars like Tesla that has auto pilot that heavily uses GPS data.<br />
<br />
<br /></td><td></td></tr>
</tbody></table>
sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com0tag:blogger.com,1999:blog-7798377.post-57865592475727501822011-03-24T19:21:00.000-07:002011-03-24T19:23:11.016-07:0050th Post- Sony HDR-XR150 Full HD 120GB HDD, 25X Optical zoom<table border="0"><tbody>
<tr><td>My Ex-Neighbor Mr. Ron asked me to suggest him a good handycam for him. Sony was the defacto brand suggestion for him. Other reasons for selecting this model.<br />
<ul><li>1920 x 1080 pixels Full High definition videos</li>
<li>It had a large 120GB space to record lots of videos</li>
<li>25X optical zoom. I never had that much of zoom so far in any of my camera/handycams.</li>
<li>Other specs like </li>
<ul><li>Exmor R" CMOS sensor</li>
<li>Smile shutter</li>
<li>Face detection</li>
</ul><li>4.5 User review ratings</li>
</ul>Not buying a memory card as it doesnt need one, it has 120GB HDD :)</td><td><div style="text-align: right;"><iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?lt1=_blank&bc1=000000&IS2=1&nou=1&bg1=FFFFFF&fc1=000000&lc1=0000FF&t=sanjkattste07-20&o=1&p=8&l=as1&m=amazon&f=ifr&ref=qf_sp_asin_til&asins=B0031RGL2A" style="height: 240px; width: 120px;"></iframe></div></td></tr>
</tbody></table>Hope he enjoys it for years to come.sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com7tag:blogger.com,1999:blog-7798377.post-24398464821096717312010-12-14T18:46:00.000-08:002014-06-13T17:51:37.578-07:00Sony TX5 Waterproof, HDIts time for another camera. Read about my earlier <a href="http://sanjaykattimani.blogspot.com/2007/06/sony-cybershot-dsc-n2-review-of.html">camera review.</a><br />
<iframe align="right" frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?t=sanjkattstech-20&o=1&p=8&l=bpl&asins=B00385XYU2&fc1=000000&IS2=1&lt1=_blank&m=amazon&lc1=0000FF&bc1=000000&bg1=FFFFFF&f=ifr" style="align: left; height: 245px; padding-right: 10px; padding-top: 5px; width: 131px;"></iframe><br />
My expectations from digital camera have increased over period of time.<br />
<br />
<strong>Reasons for concluding on this camera.</strong><br />
* Water proof camera - Hope this does not disappoint me. I also dont have any deepdive plans.<br />
* Small and very handy - Almost fits in the palm <br />
* Reliable brand- Almost every one agrees, Sony is one of the best in electronics <br />
* 10 FPs capability-Should be a good option when we know about a great shot.<br />
* 720P HD video recording - This was another must have feature i was looking for. HD is awesome on TV. <br />
* Sweep Panorama mode - Yet to really use this feature<br />
* Design - Many others with similar features didnt look as good. <br />
<br />
<iframe align="left" frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?t=sanjkattstech-20&o=1&p=8&l=bpl&asins=B003VNKNEQ&fc1=000000&IS2=1&lt1=_blank&m=amazon&lc1=0000FF&bc1=000000&bg1=FFFFFF&f=ifr" style="align: left; height: 245px; padding-right: 10px; padding-top: 5px; width: 131px;"></iframe><br />
10 MP Camera!! I didnt see any change in this from my last camera. Having water proof was a great addition compared to increased mega pixels. Also higher mega pixels does not mean better picture :) Surprisingly many cameras have always been promoting higher mega pixels and have been successful in convincing the users.<br />
<strong>Some more tips:</strong><br />
* Higher the class of your memory card, faster will be the save/retrieval operation (note the Class 10 memory card)<br />
* Compare the best price you got on this blog, amazon should automatically show you the best possible price.<br />
<br />
<iframe align="right" frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?t=sanjkattstech-20&o=1&p=8&l=bpl&asins=B0046CDGAY&fc1=000000&IS2=1&lt1=_blank&m=amazon&lc1=0000FF&bc1=000000&bg1=FFFFFF&f=ifr" style="align: left; height: 245px; padding-right: 10px; padding-top: 5px; width: 131px;"></iframe><br />
My latest <b>wish list </b>(I already miss some of these in this new camera)<br />
<br />
* DSLR quality pics (highly desirable)<br />
* 10+ optical zoom (highly desirable)<br />
* Full HD video (Desirable -Well digital cameras are meant for pics not for great video, but if it can replace video camera that would great)<br />
* Wifi Capability<br />
* Bluetooth (Low priority)<br />
* Projector? (Low priority)<br />
as usually all these in a compact camera.sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com5tag:blogger.com,1999:blog-7798377.post-4291822960293972062010-10-18T18:08:00.001-07:002010-10-18T18:33:24.166-07:00Convert Youtube to other formatsEver wondered how to play youtube video in your cd player or how to watch it offline?<br />For those of you who havent explored or asked this question, here is some gyan.<br /><br /><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYdwtwh1j_Pu0NmUysqyEfRN2K-S5nU3euNhSDnLK05w2u7QhxGqyuTaFYzj8y-5iFP24NVLd35jdCL4UGvmeYi7w5NU1AwONdXmy-h9cIOMJEp9fbQ47B2zmCDyCNd2_xvsX15A/s1600/youtube.jpg"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 104px; height: 50px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYdwtwh1j_Pu0NmUysqyEfRN2K-S5nU3euNhSDnLK05w2u7QhxGqyuTaFYzj8y-5iFP24NVLd35jdCL4UGvmeYi7w5NU1AwONdXmy-h9cIOMJEp9fbQ47B2zmCDyCNd2_xvsX15A/s400/youtube.jpg" border="0" alt=""id="BLOGGER_PHOTO_ID_5529559950731273026" /></a>Lot of freeware applications exist (do check their reliability from the reviews) that can take youtube URL (song address) and download it for you to almost any format.<br /><br /><a href="http://www.vidtomp3.com/">http://www.vidtomp3.com/</a> Converts the URL in to standard or high quality mp3 file. You can play it strait in your car(Provided you have a mp3 player in your car). Found this site very handy.<br /><br />Just search for download youtube + format you want and some website will convert it for you withought even downloading/installing any software.sanjaykattimanihttp://www.blogger.com/profile/06157889972732355536noreply@blogger.com1