Disqus - Latest Comments for shantanuo

Re: Deliver Amazon CloudWatch logs to Amazon OpenSearch Serverless

shantanuo — Mon, 12 Aug 2024 05:42:58 -0000

Thanks for this excellent article. But I think it will be easier if you make it available as an ingestion pipeline. (like CloudTrail and VPC flow log)

Re: Integrate your data and collaborate using data preparation in AWS Glue Studio

shantanuo — Tue, 16 Jul 2024 04:13:04 -0000

Thank you for this tool. I have used opneRefine before. https://github.com/OpenRefine/OpenRefine But Glue Studio is better to process big data.

Re: Ingest and analyze your data using Amazon OpenSearch Service with Amazon OpenSearch Ingestion

shantanuo — Thu, 04 Jul 2024 04:49:19 -0000

1) Is it possible to write a cloudformation template to create SQS queue, S3 bucket and IAM role mentioned in the first few steps?
2) The animated GIF files are playing at a speed that is too fast, making them difficult to comprehend.

edit: I successfully managed to ingest the data after considerable trial and error. Thank you for the excellent article.

Re: Handle tables without primary keys while creating Amazon Aurora MySQL or Amazon RDS for MySQL zero-ETL integrations with Amazon Redshift

shantanuo — Thu, 25 Apr 2024 03:49:26 -0000

Thanks for the zero-ETL integration feature. Using DMS (Data Migration Service) is expensive and complicated compared to this.
It goes without saying that every table should have a Primary Key.

Re: Amazon OpenSearch Serverless now supports automated time-based data deletion

shantanuo — Thu, 25 Jan 2024 03:13:23 -0000

Very useful information.
But there is no example even if it has been mentioned that it is possible to create a data lifecycle policy using CLI or CloudFormation.

Re: Power neural search with AI/ML connectors in Amazon OpenSearch Service

shantanuo — Thu, 25 Jan 2024 02:47:31 -0000

By default, the template deploys the Hugging Face sentence-transformers model. Can I use text-embedding-ada-002 by openai?

Re: Use Amazon Athena with Spark SQL for your open-source transactional table formats

shantanuo — Thu, 25 Jan 2024 01:39:42 -0000

Is there a cloudformation template that will take care of requirements mentioned in Prerequisites section?

Re: How to Receive Alerts When Your IAM Configuration Changes

shantanuo — Sun, 03 Sep 2023 05:18:22 -0000

is there a cloudformation template to deploy this easily?

Re: Perform upserts in a data lake using Amazon Athena and Apache Iceberg

shantanuo — Wed, 31 May 2023 04:53:23 -0000

very nice article. Thank you for the step by step guide.
But got an error mismatched input '<eof>'. Expecting: '%', ')', '*', in last statement.... MERGE INTO curated_demo.sporting_event t USING (SELECT op, ...

Re: Debug AWS DMS tasks using Time Travel

shantanuo — Wed, 14 Sep 2022 03:31:25 -0000

Time Travel seems to be available for PostgreSQL to either PostgreSQL or MySQL. I will like to see sql-server to MySQL support. Is that possible in the future?

Re: Supercharging Dream11’s Data Highway with Amazon Redshift RA3 clusters

shantanuo — Tue, 28 Jun 2022 04:00:24 -0000

Nice article. But you have mentioned "the newer version of the automated AWS CloudFormation-based toolset (now on GitHub), was not available." and it seems that the links are not working.

Re: Optimize performance and reduce costs for network analytics with VPC Flow Logs in Apache Parquet format

shantanuo — Sat, 08 Jan 2022 02:50:21 -0000

Thanks for the article. It works as expected. But I have a question...
https://stackoverflow.com/q...

Re: Improve Amazon Athena query performance using AWS Glue Data Catalog partition indexes

shantanuo — Fri, 07 Jan 2022 04:17:29 -0000

Thanks for your reply. One more question. Once you add data for the year 2022 to S3, can I query using Athena?

Re: What’s new in Amazon Redshift – 2021, a year in review

shantanuo — Tue, 04 Jan 2022 02:48:50 -0000

Redshift announced support for Lambda UDFs in Oct 2020. It was not in the year 2021, but worth a mention! https://aws.amazon.com/abou...

Re: Extending Pandas | Dr. Bryan Patrick Wood's Website

shantanuo — Fri, 24 Dec 2021 23:41:24 -0000

Nice article. But there is a typo:
The output as mentioned in the article is wrong.

0 zAR 1 zAZ

It shoud be Z (capital Z) and not small z

Re: Improve Amazon Athena query performance using AWS Glue Data Catalog partition indexes

shantanuo — Tue, 14 Dec 2021 00:01:53 -0000

What ill be the charges if I complete all the steps mentioned in this tutorial?

Re: Introducing Amazon Redshift Serverless – Run Analytics At Any Scale Without Having to Manage Data Warehouse Infrastructure

shantanuo — Sun, 05 Dec 2021 06:20:26 -0000

It is mentioned in the article that "To control your costs, you can specify usage limits and define actions that Amazon Redshift automatically takes." But I can not find that option from console.

Re: Choosing between storage mechanisms for ML inferencing with AWS Lambda

shantanuo — Wed, 24 Nov 2021 03:29:12 -0000

awesome post. But I have a doubt. what if it takes more than 30 seconds to return the results? will it timeout?

Re: Use pre-trained financial language models for transfer learning in Amazon SageMaker JumpStart

shantanuo — Sat, 09 Oct 2021 06:56:28 -0000

Thanks for sharing this. I will certainly try it. But how much does it (sagemaker endpoint) cost?

Re: Hosting Hugging Face models on AWS Lambda for serverless inference

shantanuo — Fri, 17 Sep 2021 02:41:12 -0000

Thanks for this very useful article. But can you also mention the cost?

Re: Dynamic image resizing with Python and Serverless framework

shantanuo — Fri, 02 Jul 2021 03:17:55 -0000

This is very interesting. But you should make it more developer friendly. There is a "Suggest a Bot" link, but that is not enough. A programmer should be able to submit his docker image as a new bot.

Re: Accessing external components using Amazon Redshift Lambda UDFs

shantanuo — Thu, 29 Oct 2020 00:59:02 -0000

Awesome. Thanks for your help. If in case my lambda function takes time for e.g. 15 to 20 minutes, How is that handled by redshift?

Re: Accessing external components using Amazon Redshift Lambda UDFs

shantanuo — Wed, 28 Oct 2020 23:56:04 -0000

Can you suggest how to rewrite the lambda function code if it looks like this...
https://gist.github.com/sha...

I need to count the number of input variables and return the same number of results those are returned from that API.

Re: Accessing external components using Amazon Redshift Lambda UDFs

shantanuo — Wed, 28 Oct 2020 09:37:49 -0000

Interesting. I tried it and have a question that I asked on stack overflow.

https://stackoverflow.com/q...

Re: Analyzing Amazon S3 server access logs using Amazon ES

shantanuo — Sun, 25 Oct 2020 05:32:03 -0000

Thanks for sharing this. But I am getting DeprecationWarning.

You are using the put() function from 'botocore.vendored.requests'. This dependency was removed from Botocore and will be removed from Lambda after 2021/01/30. Install the requests package, 'import requests' directly, and use the requests.put() function instead.