https://cloudsqale.com 2026-11-07T02:50:00+00:00 1.00 https://cloudsqale.com/about/ https://cloudsqale.com/2023/11/26/spark-order-by-implementation/ https://cloudsqale.com/category/data-skew/ https://cloudsqale.com/category/optimizer/ https://cloudsqale.com/category/spark/ https://cloudsqale.com/author/dmtolpeko/ https://cloudsqale.com/2023/10/25/spark-reading-json-sampling/ https://cloudsqale.com/category/i-o/ https://cloudsqale.com/category/json/ https://cloudsqale.com/2023/10/15/distributed-count-distinct-how-it-works-in-spark-multiple-count-distinct-transform-to-count-with-expand-exploded-shuffle-partial-aggregations/ https://cloudsqale.com/2023/10/10/spark-reading-parquet-pushed-filters-substrtimestamp-1-10-like-and-stringstartswith/ https://cloudsqale.com/2023/10/06/spark-stage-restarts-partial-restarts-multiple-retry-attempts-with-different-task-sets-accepted-late-results-from-failed-stages-cost-of-restarts/ https://cloudsqale.com/2023/10/01/spark-aqe-stage-numeration-added-jobs-at-runtime-large-number-of-tasks-pending-and-skipped-stages/ https://cloudsqale.com/2023/09/17/spark-limit-on-large-datasets-collectlimit-globallimit-locallimit-spark-sql-limit-scaleupfactor/ https://cloudsqale.com/2023/07/19/spark-number-of-tasks-reading-large-number-of-small-parquet-files/ https://cloudsqale.com/category/parquet/ https://cloudsqale.com/2022/08/30/spark-2-4-slow-performance-on-writing-into-partitions-why-sorting-involved/ https://cloudsqale.com/category/storage/ https://cloudsqale.com/2022/08/30/spark-create-multiple-output-files-per-task-using-spark-sql-files-maxrecordsperfile/ https://cloudsqale.com/page/2/ https://cloudsqale.com/2023/11/ https://cloudsqale.com/2023/10/ https://cloudsqale.com/2023/09/ https://cloudsqale.com/2023/07/ https://cloudsqale.com/2022/08/ https://cloudsqale.com/2022/04/ https://cloudsqale.com/2021/03/ https://cloudsqale.com/2021/01/ https://cloudsqale.com/2020/06/ https://cloudsqale.com/2020/05/ https://cloudsqale.com/2020/04/ https://cloudsqale.com/2020/02/ https://cloudsqale.com/2019/12/ https://cloudsqale.com/2019/11/ https://cloudsqale.com/2019/10/ https://cloudsqale.com/2019/09/ https://cloudsqale.com/2019/08/ https://cloudsqale.com/2019/05/ https://cloudsqale.com/2019/04/ https://cloudsqale.com/2019/01/ https://cloudsqale.com/2018/12/ https://cloudsqale.com/2018/11/ https://cloudsqale.com/2018/10/ https://cloudsqale.com/2018/09/ https://cloudsqale.com/category/amazon/ https://cloudsqale.com/category/auto-scaling/ https://cloudsqale.com/category/aws/ https://cloudsqale.com/category/cost-optimization/ https://cloudsqale.com/category/cpu/ https://cloudsqale.com/category/ec2/ https://cloudsqale.com/category/emr/ https://cloudsqale.com/category/etl/ https://cloudsqale.com/category/flink/ https://cloudsqale.com/category/hadoop/ https://cloudsqale.com/category/hive/ https://cloudsqale.com/category/hue/ https://cloudsqale.com/category/jvm/ https://cloudsqale.com/category/kinesis/ https://cloudsqale.com/category/logs/ https://cloudsqale.com/category/memory/ https://cloudsqale.com/category/monitoring/ https://cloudsqale.com/category/orc/ https://cloudsqale.com/category/pig/ https://cloudsqale.com/category/presto/ https://cloudsqale.com/category/qubole/ https://cloudsqale.com/category/rds/ https://cloudsqale.com/category/s3/ https://cloudsqale.com/category/snowflake/ https://cloudsqale.com/category/tez/ https://cloudsqale.com/category/yarn/ https://cloudsqale.com/2019/01/25/reduce-number-of-output-files-for-skewed-data-order-in-apache-pig-sampler-and-weighted-range-partitioner-to-balance-reducers/ https://cloudsqale.com/category/spark/page/2/ https://cloudsqale.com/author/dmtolpeko/page/2/ https://cloudsqale.com/2021/03/19/spark-reading-parquet-why-the-number-of-tasks-can-be-much-larger-than-the-number-of-row-groups/ https://cloudsqale.com/2021/03/07/spark-reading-parquet-predicate-pushdown-for-like-operator-equalto-startswith-and-contains-pushed-filters/ https://cloudsqale.com/2020/06/18/how-map-column-is-written-to-parquet-converting-json-to-map-to-increase-read-performance/ https://cloudsqale.com/2020/06/09/flink-streaming-to-parquet-files-in-s3-massive-write-iops-on-checkpoint/ https://cloudsqale.com/2020/06/04/s3-low-latency-writes-using-aggressive-retries-to-get-consistent-latency-request-timeouts/ https://cloudsqale.com/category/i-o/page/2/ https://cloudsqale.com/2021/01/15/parquet-1-x-file-format-footer-content/ https://cloudsqale.com/2020/05/29/how-parquet-files-are-written-row-groups-pages-required-memory-and-flush-operations/ https://cloudsqale.com/2019/12/30/spark-slow-load-into-partitioned-hive-table-on-s3-direct-writes-output-committer-algorithms/ https://cloudsqale.com/2019/05/14/snowflake-monitoring-data-ingestion-using-query_history-and-copy_history-single-large-file-vs-multiple-small-files/ https://cloudsqale.com/2018/12/26/storage-tuning-for-mapped-json-conversion-to-orc-file-format-java-heap-issues-with-dictionary-encoding/ https://cloudsqale.com/2018/12/17/orc-file-format-internals-creating-large-stripes-in-hive-tables/ https://cloudsqale.com/2018/10/18/s3-monitoring-4-read-operations-and-tables/ https://cloudsqale.com/2018/10/10/s3-monitoring-step-3-read-operations-and-file-types/ https://cloudsqale.com/2018/10/09/s3-monitoring-step-2-read-operations/ https://cloudsqale.com/category/storage/page/2/ https://cloudsqale.com/2022/08/29/emr-spark-initial-number-of-executors-and-spark-dynamicallocation-enabled/ https://cloudsqale.com/2022/08/26/emr-spark-much-larger-executors-are-created-than-requested/ https://cloudsqale.com/2022/04/20/amazon-emr-spark-ignoring-partition-filter-and-listing-all-partitions-when-reading-from-s3a/ https://cloudsqale.com/2021/01/02/flink-and-s3-entropy-injection-for-checkpoints/ https://cloudsqale.com/2020/06/25/hadoop-yarn-monitoring-resource-consumption-by-running-applications-in-multi-cluster-environments/ https://cloudsqale.com/page/3/ https://cloudsqale.com/2020/05/27/s3-multipart-upload-5-mb-part-size-limit/ https://cloudsqale.com/2020/05/26/flink-s3-checkpoints-monitoring-using-s3-access-logs/ https://cloudsqale.com/2020/05/26/hive-table-for-s3-access-logs/ https://cloudsqale.com/2020/05/20/kinesis-client-library-kcl-2-x-consumer-load-balancing-rebalancing-taking-renewing-and-stealing-leases/ https://cloudsqale.com/2020/05/08/yarn-negative-vcores-capacity-scheduler-with-memory-resource-type/ https://cloudsqale.com/2020/05/07/aws-ec2-vcpu-and-yarn-vcores-m4-c4-r4-instances/ https://cloudsqale.com/2020/05/02/s3-rest-api-http-1-1-requests-for-uploading-files/ https://cloudsqale.com/2020/04/17/s3-multipart-upload-s3-access-log-messages/ https://cloudsqale.com/2020/04/29/flink-1-9-off-heap-memory-on-yarn-troubleshooting-container-is-running-beyond-physical-memory-limits-errors/ https://cloudsqale.com/2020/04/12/flink-tuning-writes-to-s3-sink-fs-s3a-threads-max/ https://cloudsqale.com/2020/02/26/hive-on-tez-shuffle-failed-with-too-many-fetch-failures-and-insufficient-progress/ https://cloudsqale.com/2020/02/19/hadoop-yarn-container-virtual-memory-understanding-and-solving-container-is-running-beyond-virtual-memory-limits-errors/ https://cloudsqale.com/2019/01/21/tez-memory-tuning-container-is-running-beyond-physical-memory-limits-solving-by-reducing-memory-settings/ https://cloudsqale.com/2020/02/14/hadoop-yarn-cluster-idle-time/ https://cloudsqale.com/2019/09/11/hadoop-yarn-calculating-per-second-utilization-of-cluster-using-resource-manager-logs/ https://cloudsqale.com/2019/12/11/hive-issues-with-large-yarn-containers-low-concurrency-and-utilization-high-execution-time/ https://cloudsqale.com/2019/12/04/presto-query-auto-scaling-limitations-low-utilization-after-upscale-group-by-partitioned-stage-query-manager-required-workers/ https://cloudsqale.com/2019/12/04/presto-vs-hive-sla-risks-for-long-running-etl-failures-and-retries-due-to-node-loss/ https://cloudsqale.com/2019/12/02/snowflake-micro-partitions-and-clustering-depth/ https://cloudsqale.com/2019/05/03/performance-of-min-max-functions-metadata-operations-and-partition-pruning-in-snowflake/ https://cloudsqale.com/2019/11/28/presto-troubleshooting-query-exceeded-per-node-total-memory-limit-resource_overcommit-query-max-total-memory-per-node-reserved-pool-disk-spill/ https://cloudsqale.com/2019/11/20/apache-hive-on-tez-quick-on-the-fly-profiling-of-long-running-tasks-using-jstack-probes-and-flame-graphs/ https://cloudsqale.com/2019/11/16/apache-hive-monitoring-progress-of-long-running-reducers-hive-log-every-n-records-option/ https://cloudsqale.com/2019/11/01/apache-hive-pig-on-tez-long-running-tasks-and-their-failed-attempts-analyzing-performance-and-finding-bottlenecks-insufficient-parallelism-using-application-master-logs/ https://cloudsqale.com/2019/10/18/sqoop-import-from-amazon-rds-read-replica-error-canceling-statement-due-to-conflict-with-recovery/ https://cloudsqale.com/2019/05/15/hadoop-yarn-collecting-utilization-metrics-from-multiple-clusters/ https://cloudsqale.com/2019/09/04/tuning-hadoop-yarn-boosting-memory-settings-beyond-the-limits-to-increase-cluster-capacity-and-utilization/ https://cloudsqale.com/2019/08/27/amazon-emr-downscaling-and-ghost-impaired-nodes/ https://cloudsqale.com/2019/05/23/amazon-emr-recovering-ghost-nodes/ https://cloudsqale.com/2019/05/23/amazon-emr-recovering-unhealthy-nodes-with-emr-services-down/ https://cloudsqale.com/2019/05/20/amazon-emr-monitoring-auto-scaling-using-instance-controller-logs/ https://cloudsqale.com/2019/05/11/snowflake-data-ingestion-cluster-utilization-idle-time-and-compute-cost/ https://cloudsqale.com/2019/05/06/snowflake-reloading-data-from-stage-truncate-delete-copy-and-transactions/ https://cloudsqale.com/2019/05/04/snowflake-remote-disk-i-o-local-disk-cache-capacity-utilization-and-transfer-rate/ https://cloudsqale.com/2019/04/19/yarn-memory-under-utilization-running-low-memory-instances-c4-xlarge-i-e/ https://cloudsqale.com/2019/04/08/extremely-large-number-of-rdd-partitions-and-tasks-in-spark-on-amazon-emr/ https://cloudsqale.com/2019/01/04/yarn-resource-manager-silent-restarts-java-heap-space-error-amazon-emr/ https://cloudsqale.com/2018/12/24/orc-files-split-computation-hive-on-tez/ https://cloudsqale.com/2018/11/12/tez-internals-2-number-of-map-tasks-for-large-orc-files-with-small-stripes-in-amazon-emr/ https://cloudsqale.com/2018/12/04/s3-writes-when-inserting-data-into-a-hive-table-in-amazon-emr/ https://cloudsqale.com/2018/10/22/tez-internals-1-number-of-map-tasks/ https://cloudsqale.com/2018/10/08/s3-monitoring-step-1-bucket-size-and-number-of-objects/ https://cloudsqale.com/2018/09/25/collecting-s3-access-logs/ https://cloudsqale.com/2018/10/05/simple-hive-queries-with-predicates-compressed-text-vs-orc-files/ https://cloudsqale.com/2018/09/26/operation-timed-out-for-hive-query-in-hue-and-qubole/ https://cloudsqale.com/category/amazon/page/2/ https://cloudsqale.com/category/aws/page/2/ https://cloudsqale.com/category/emr/page/2/ https://cloudsqale.com/category/hadoop/page/2/ https://cloudsqale.com/category/hive/page/2/ https://cloudsqale.com/category/s3/page/2/ https://cloudsqale.com/category/yarn/page/2/ https://cloudsqale.com/author/dmtolpeko/page/3/ https://cloudsqale.com/category/i-o/page/3/ https://cloudsqale.com/category/aws/page/3/