Dask Demo Day - Sign up issue

[ncclementi]

Next Demo Day: December 5th

See what the Dask community has been up to, or share some Dask work of your own. Demos are short and informal (~5-10 minutes). Have something you'd like to share? Leave a comment and let us know.

Meetings are held on the first Thursday of each month at 11am US Eastern on Zoom: https://dask.org/meeting-room

Subscribe to the Dask event calendar to be notified of changes:

• Google Calendar
• iCal

Past Demo Days

Meetings are recorded and shared on the Dask YouTube channel.

Oct 3rd, 2024

• @rjzamora "pin and persist"

• Sept 5th, 2024 youtube recording

  • @dougbrn Dask Applications in Astronomy with LINCC Frameworks
  • @jacobtomlinson dask-jobqueue demo

• August 1st, 2024

  • @raybellwaves github-repo-explorer

• June 6th, 2024

  • @ahuang11 streamjoy
  • @v2thegreat hvPlot

• March 21st, 2024 youtube recording

  • @fjetter Dask DataFrame improvements
  • @mrocklin Large scale population of vector databases for RAG
  • @jrbourbeau Easy GPU access with Coiled

• February 15th, 2024 youtube recording

  • @mrocklin One trillion row challenge
  • @jacobtomlinson deploying Dask on Databricks with dask-databricks
  • @jrbourbeau deploying Prefect workflows on the cloud with Coiled
  • @quasiben scaling embedding pipelines (LlamaIndex + Dask)
  • @ntabris using AWS Cost Explorer to see the cost of public IPv4 addresses

• January 18th, 2024 youtube recording

  • @cisaacstern (and maybe @jacobtomlinson) Apache Beam DaskRunner
  • @mrocklin Array expressions
  • @scharlottej13 one billion row challenge

• October 19th, 2023 youtube recording

  • @mrocklin TPC-H benchmarks for Spark, Dask, Polars, DuckDB
  • @mrchtr Fondant
  • @jhamman Dask <> Arraylake integration
  • @jacobtomlinson "Who uses RAPIDS?"

• September 21st, 2023 youtube recording

  • @fjetter Performance with P2P array rechunking
  • @phofl Dask expressions
  • @scharlottej13 + @dcherian Processing a quarter petabyte geospatial dataset in the cloud

• August 17th, 2023 youtube recording

  • @fjetter -- distributed + memray integration for memory profiling
  • @mrocklin -- coiled setup CLI
  • @jrbourbeau -- Make Dask and Earthaccess work well together

• July 20th, 2023 youtube recording

  • @hendrikmakait Shuffle resilience
  • @Matt711 : Dask-Kubernetes update
  • @GueroudjiAmal external tasks in Dask distributed (https://github.com/GueroudjiAmal/distributed)
  • @skrawcz Dask <> Hamilton integration

• June 15th, 2023 youtube recording

  • dask-geopandas demo by @martinfleis
  • Fine performance dask metrics @crusaderky
  • Gil monitoring on dask @milesgranger

• May 18th, 2023 youtube recording

  • Dask Expressions - @rjzamora
  • deltadask - @MrPowers
  • ydata-profiling - @miriamspsantos / @fabclmnt
  • Dask-bigquery - @ncclementi /@j-bennet

• April 20th, 2023 -youtube recording

  • dask-awkward and dask-histogram for high energy physics analysis @lgray (10 min)
  • daskqueue : a dask-based distributed task queue. @AmineDiro (5-7min)
  • Pyarrow strings in Dask DataFrames - @jrbourbeau (5-7 min)
  • Launching a Jupyter/Dask cluster on NVIDIA Base Command Platform - @jacobtomlinson (5-7min)

• March 16th, 2023 - youtube recording

  • Analysing Terabytes of Ocean Simulation model output with Xarray, xgcm and xhistogram @TomNicholas
  • P2P shuffling @hendrikmakait
  • Scaling weather radar data analysis with Dask @mgrover1
  • Automatic package synchronization in Coiled Dask Clusters @dchudz
  • Graph Neural Networks training with Dask @VibhuJawa

[ncclementi]

• @brian-methodical Can we get some folks from your team to sign up for one of the upcoming dates?

• @mgrover1 Do you think the 03/16 date would work for you? Do you want to drop a comment with a proposed date and title?

• @TomNicholas Whenever you can drop a comment on what date would be more convenient for you, that would be great.

• @raybellwaves @jorisvandenbossche @tastatham @martinfleis Can we get a demo (or two) that involves dask-geopandas? Or can you volunteer someone :) ?

[martinfleis]

@ncclementi I could do a short demo of dask-geopandas in May but happy to leave the place to others if they're interested.

[ncclementi]

@martinfleis I'll take you up for May, if you know of folks that would like to show their applications on any of the dates, please send them this issue. It'll be great to hear more about how dask-geopandas is being used.

[TomNicholas]

I think 16th March is good for me!

[ncclementi]

@TomNicholas Thanks Tom 🙌 , do you have a title that would summarize your demo? If not yet, we can set it later

[TomNicholas]

Something like "Analysing Terabytes of Ocean Simulation model output with Xarray, xgcm and xhistogram"

[dchudz]

@ncclementi I'd like to demo Coiled's "package sync" feature (should be interesting even for folks not planning to use Coiled).

[mgrover1]

Yes to 3/16! Still trying to come up with a title, but it will be related to scaling weather radar data analysis with dask!

[ncclementi]

Thanks @mgrover1 and @dchudz We'll have you on the list! 🎉

[ncclementi]

@martindurant and @douglasdavis would you be interested in presenting at the April upcoming session?

[lgray]

Hi @ncclementi - I'm using dask-awkward and dask-histogram for high energy physics analysis and can give some real-world examples of how we'd use @martindurant and @douglasdavis's work. I think I can fit that all into 10 minutes. I'm available for April 20th.

[ncclementi]

@lgray That would be awesome! We usually do 5-7 min but since we do not have folks for April yet, I think 10 min will work, plus you will be showcasing two dask projects!
I'll write you down for April 20th then!!

[AmineDiro]

Hello @ncclementi , I built daskqueue, a lightweight Distributed Task Queue library built on top of Dask. Daskqueue also implements persistent queues for holding tasks on disk and surviving Dask cluster restart. I would love to present this library according to the format of your choosing and April 20th sounds excellent for me.

The title of the talk would be daskqueue : a dask-based distributed task queue.