avatarChristianlauer

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

3902

Abstract

only, some have video only, and some have both video and audio. And also they’re available in multiple formats. For video there is AVC (which is apparently H.254), VP09, and AV01. And for audio there’s Mp4a and Opus. I don’t know what Opus is but apparently it is noticeably better than Mp4a. So if you’ve ever gone to a YouTube download page and seen a million download options that’s why. Although I hide the redundant ones and just display the one with the lowest file size at that quality level (usually VP09 or AV01).</p><p id="b96b">It’s actually pretty cool to see AV01 as I talked about it <a href="https://readmedium.com/why-apple-adding-av1-support-is-a-big-deal-faa46a356f3c">here</a>. And it does compress videos a little better than VP09 and a lot better than AVC.</p><p id="d170">Also apparently YouTube does something called DRC, dynamic range compression (Not Democratic Republic of Congo). It makes the quiet sections louder and the loud sections quieter. Apparently people really hate it and I guess there must be a way to disable it because I have not been able to find a music video with DRC enabled. I’ve never heard anyone talk about this before though.</p><p id="e027">So now that we have the formats we can just download them. Each format has a download link oddly enough. I did not expect that. Just keep in mind that the YouTube download links expire in 6 hours.</p><p id="3fc9">Of course here we run into a problem. When I saw this I was like, “Great, let’s just download an audio-only track if we only want audio or a video+audio track if we want video.”</p> <figure id="dd49"> <div> <div> <img class="ratio" src="http://placehold.it/16x9"> <iframe class="" src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FMvioOPqhOm6b4BI0FQ%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Ffallontonight-jimmy-fallon-tonight-show-MvioOPqhOm6b4BI0FQ&amp;image=https%3A%2F%2Fmedia0.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExa203NTd1dnVhNGpveHByMmxsOGRzZGxsejhsZzh2Zjk5N20xMTYwbSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FMvioOPqhOm6b4BI0FQ%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" allowfullscreen="" frameborder="0" height="435" width="435"> </div> </div> </figure></iframe></div></div></figure><p id="9eda">If you do that you’re restricted to at most 720p 30fps. Or so I’m told. I haven’t been able to find a file with both video and audio over 360p. And some videos don’t have any video+audio tracks at all. So that means we’re going to have to download a video file and an audio file (preferably without DRC) and merge them together.</p><p id="e01b">And this is perhaps the first really big problem I ran into. If I was doing this locally it would not be that big of a problem because there’s a nice command line package called <code>FFmpeg</code> that will just do it for you. Unfortunately it’s a bit more complicated when you’re working with Node.js because you have to do a bunch of file manipulation stuff.</p><p id="12d8">I ended up deciding to just download the files from Google’s servers separately and then merge them locally. Apparently Flutter has a package called <code>FFmpeg-kit</code> that will do this. Unfortunately it only works on Android, iOS, and MacOS.</p><p id="fdbf">Although Android will not be getting YouTube downloading because Google is really strict about that.</p> <figure id="eaa2"> <div> <div> <img class="ratio" src="http://placehold.it/16x9"> <iframe class="" src="https://cdn.embedly.com/widgets/media.html?type=text%2Fhtml&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;schema=twitter&amp;url=https%3A//twitter.com/EzraDuown/status/1776352074449530961/&amp;image=" a

Options

llowfullscreen="" frameborder="0" height="281" width="500"> </div> </div> </figure></iframe></div></div></figure><p id="7c7c">In fact the app mentioned above, NewPipe, is not available on the Google Play store. So that means that the only platform able to benefit from YouTube downloading is iOS. There is no macOS version of the app but you can run the iOS app on an Apple Silicon Mac. Web users will be restricted to downloading audio or downloading 360p videos. Oh well. Maybe I’ll improve this feature in the future.</p><p id="cae5">And another problem. YouTube throttles some of the downloads. Not all of them though which is weird. I thought it was broken at first. Nope, it’s just that the downloads were really really slow.</p><p id="74fe">Although there’s a way around this too. Flutter has its own package for downloading YouTube videos called <code>youtube_explode_dart</code> and this one has gotten around the throttling issues. It’s listed as issue #180 and fixed in pull request 185.</p><figure id="c76e"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*R0QbSVVKcpNt1NDnlL-7Qg.png"><figcaption></figcaption></figure><p id="4a2c">And it works great now. Well, for the streams it supports. It doesn’t support all the streams. But it supports enough.</p><p id="7308">And, one more thing <code>youtube_explode_dart</code> doesn’t actually do everything <code>youtube-dl</code> does. <code>youtube-dl</code> actually gives you a lot more information on the actual files (such as if the audio is DRCed). So I’m still going to have to use it. But I can use <code>youtube_explode_dart</code> to actually download the files.</p><p id="8fe6">I do feel a little silly for wasting so much time on this feature. Like, I built an RSS reader to watch YouTube videos, not to download them. And the only people that can really take advantage of this feature are iOS users.</p><p id="9388">It’s funny. Everyone thinks iOS is the locked-down one. But in this one specific case Android is more locked down. Because if you look up YouTube downloaders on iOS there are pages and pages of them.</p><p id="0e6e">I could see myself taking advantage of this feature. Stratum 3.0.0 also introduces the ability to add external links. So I could find a YouTube video I like that I want to download and I could just download it.</p><p id="01c9">And this experience has taught me a lot about the inner workings of YouTube. Mostly the subtitles. I had no idea the method I was using to get subtitles before was so suboptimal. It works and there’s nothing wrong with it but it could be better.</p><p id="7e7d">I’m pretty happy with Stratum’s YouTube functionality now. I was happy with it before, to be honest, I just went down a ‘YouTube Rabbit Hole’. But I did have one more feature planned: automatic YouTube summary generation.</p><p id="5b0e">Because since adding YouTube summaries I’ve been watching a lot more YouTube. I mean summarizing a lot more YouTube. There are a lot of channels I’m interested in but can’t get to due to time restrictions. Like TED and DW News. Guess what? I follow both of those sources of news now and just read the summaries. But you know what I don’t like? Tapping on the button to generate the summaries.</p><p id="8a38">Well, those are some future plans. It will be expensive though, which is also why I’m planning to announce new pricing for Stratum soon.</p><p id="f6db">If you liked this post and would like to stay updated with my future articles consider using my RSS app Stratum on <a href="https://apple.co/3rZyh9B">iOS</a> and <a href="https://play.google.com/store/apps/details?id=com.amorfatite.keystone">Android</a>. Also check out my language learning app Litany (<a href="https://apple.co/45prCDA">iOS</a>, <a href="https://play.google.com/store/apps/details?id=com.amorfatite.litany">Android</a>).</p></article></body>

Microsoft enforces it’s Self Service BI Strategy

How Power BI Data Marts should drive Data Driven Culture

Photo by Surface on Unsplash

With Power BI Data Mart, Microsoft wants to enable customers to gain insights from data faster and without IT.

Microsoft had already published some interesting updates around Power BI in the past weeks, some of which I was particularly happy about and wrote a small article about.

In addition to these smaller and sometimes larger updates, however, one notices that the topic of self-service BI with the Data Marts tool is also clearly focused on here. Microsoft Power BI Datamarts is a fully managed SQL database that allows users to easily perform relational database analytics using no-code or low-code [1].

Microsoft is now following up with some improvements. In addition to new regions, in this case South East Asia, new connectors are also available, for example for:

  • Amazon Athena
  • Asana
  • Azure Databricks and Databricks
  • Google Sheets
  • MariaDB
  • SurveyMonkey

In addition to these somewhat major technical improvements, some enhancements to the user experience and features that support the user in creating data marts have also become available — these can be found in the official statement from Microsoft [1].

Also likely to be a step forward for the whole Self Service BI theme is the also recently announced move to better integrate Power BI with the Mobile Teams app. This means that analyses from mobile devices will also be better supported.

However, as genious this step by Microsoft may be in the face of competition from Google, AWS, and others might be, it naturally also involves a risk. After all, modern data platforms such as Data Lakes and Data Lakehouses are supposed to limit data silos and shadow IT. This means that data is stored, monitored and shared centrally with the help of IT and data teams. Often, data marts are also created for individual areas, but with the involvement of specialized experts.

Data Lakehouse Concept — Image from Author

Now, however, departments can create their own data marts. The advantage of this is that goals are often achieved more quickly, but the disadvantage is that mistakes may be made or sources may be included that are then only known to the respective business. Of course, this could lead to problems if these sources no longer function as usual or other departments do not see this data because it is not shown in a data catalog.

To avoid this, it is advisable to work together with data and IT teams or to employ experts in the business departments to monitor these data processes. In this way, faster results can be produced, but these can also be monitored and, in a later step, perhaps also standardized and made more useful for the entire company.

Sources and Further Readings

[1] Microsoft Power BI Blog, Announcing improvements to Datamart Query editor (2022)

[2] Microsoft Power BI Blog, What’s New in Datamarts — August 2022 (2022)

Data Science
Microsoft
Technology
Power Bi
Data
Recommended from ReadMedium