Groundwork PR

1. Adds the hop
2. Adds the quack impl
3. Only supports pointwise
4. no aux buffers
5. All this to make follow up prs easier to review

sanity check

Notes for reviewer

API;

# Do be made public later
 from torch._higher_order_ops import flex_gemm

 out = flex_gemm(
     gemm_op,          # torch.mm / torch.addmm / aten op overload expanded to a wider set scaled_mm, grouped friends
     gemm_args,        # tuple of tensor/scalar operands orignal args ot he base func
     epilogue_fn,      # Python fn over accumulator/result
     *,
     gemm_kwargs=None, # non-tensor kwargs only
     kernel_options={},  # dict of options, will be backend, fastmath, tune
 )

Pull Request resolved: #186483
Approved by: https://github.com/mlazos, https://github.com/eellison
ghstack dependencies: #186944

Pointed quack at my upstream fork, pointing back to main and using patchsets as intended

Pull Request resolved: #186944
Approved by: https://github.com/slayton58

What does this do;

1. .github/workflows/quack-vendor-reproducibility.yml add a ci test that make sure the final checked in code is sha + patch set applied
2. fix patches from update
3. puts the sha as source of truth in the vendor.sh
4. adds gitattributes to minimize by default the vendored code -> makes easier to review
5. expand vendor.sh to cover gemms need for flex gemm

Pull Request resolved: #186284
Approved by: https://github.com/slayton58

Structurally similar to quack's small-K topk, but lossless: keys carry
the full original index so output is bit-exact to aten on values with
deterministic tie order, at the cost of lower throughput.

Dispatch in cutedsl_impl.py:

• K=16, N pow2 in [64, 2048]: register kernel
• K=32, N=256: register kernel
• K in {64..1024} with existing radix N gates: radix
• else: aten

On B200: K=16 2.24x geomean (1.16-4.45x), K=32 1.04x geomean
(1.00-1.45x), no regressions across the eligible (K, N, M) grid.

Test plan:

python test/python_native/test_topk_cutedsl.py

Authored by Claude.
Pull Request resolved: #184927
Approved by: https://github.com/ngimel
ghstack dependencies: #184926

Two cases slip through that shouldn't:

• ROCm builds (e.g. MI300x). get_device_capability on gfx942 returns major=9, matching the H100 check, so the dispatcher routes to Quack - a CUTLASS/CUDA-only kernel that has no business running on AMD.
• CPU tensors. _is_supported never checks input.device.type. When a CPU tensor is passed (e.g. during the FX Interpreter.run in splitting/utils.py:remove_unexpected_type_cast), get_device_capability reports the current CUDA device's capability, the cond passes, and the kernel crashes with ValueError: Mismatched Tensor ... expected device_type=cuda.

Test Plan: This fixed 3 tests internally

Differential Revision: D106733305

Pull Request resolved: #185644
Approved by: https://github.com/desertfire

Reverted #182108 on behalf of https://github.com/pytorch-auto-revert due to Reverted automatically by pytorch's autorevert, to avoid this behaviour add the tag autorevert: disable (comment)

Reverted #182108 on behalf of https://github.com/pytorch-auto-revert due to Reverted automatically by pytorch's autorevert, to avoid this behaviour add the tag autorevert: disable (comment)

Add fused rmsnorm fwd/bwd implementations from quack, written in
cutedsl. Rely on vendored quack.

Add testing to OpInfo for composability with other PyT systems, and
test_nn to match the existing fused norm tests.

Test Plan:

  python -m pytest -v \
    test/test_nn.py -k rmsnorm \
    test/python_native/ \
    test/test_ops.py -k 'rms_norm' \
    test/inductor/test_torchinductor_opinfo.py -k 'rms_norm'

  # Then separately, since OPINFO_RESTRICT_TO_DSL filters op_db globally:
  OPINFO_RESTRICT_TO_DSL=cutedsl python -m pytest -v \
    test/test_ops.py -k 'rms_norm and cutedsl' \
    test/inductor/test_torchinductor_opinfo.py -k 'rms_norm and cutedsl' \
    test/export/test_export_opinfo.py -k 'rms_norm and cutedsl'

Authors:

Originally from #178326

Co-authored-by: AaronWang04

Signed-off-by: Simon Layton simonlayton@meta.com

Pull Request resolved: #182108
Approved by: https://github.com/drisspg

Co-authored-by: AaronWang04 aaronw23880@gmail.com

“The man who sees absolutes, where all other men see nuances and shades of meaning, is either a prophet, or a quack.

Donald H. Rumsfeld is not a prophet.”

A very powerful signoff from Keith Olbermann, host of NBC “Countdown”. Read the transcript here, or watch the video in wmv or qt format.

Managed languages facilitate convenient ways for serializing objects, allowing applications to persist and transfer them easily, yet this feature opens them up to attacks. By manipulating serialized objects, attackers can trigger a chained execution of existing code segments, using them as gadgets to form an exploit. Protecting deserialization calls against attacks is cumbersome and tedious, leading to many developers avoiding deploying defenses properly.

We present QUACK, a framework for automatically protecting applications by fixing calls to deserialization APIs. This "binding" limits the classes allowed for usage in the deserialization process, severely limiting the code available for (ab)use as part of exploits. QUACK computes the set of classes that should be allowed using a novel static duck typing inference technique. In particular, it statically collects all statements in the program code that manipulate objects after they are deserialized, and puts together a filter for the list of classes that should be available at runtime. We have implemented QUACK for PHP and evaluated it on a set of applications with known CVEs and popular applications crawled from GitHub. QUACK managed to fix the applications in a way that prevented any attempt at automatically generating an exploit against them, by blocking, on average, 97% of the application's code that could be used as gadgets. We submitted a sample of three fixes generated by QUACK as pull requests, and their developers merged them.

By:
Neophytos Christou | PhD Candidate, Brown University
Andreas Kellas | Security Researcher, Columbia University

Presentation Materials Available at:
https://blackhat.com/us-25/briefings/schedule/?#quack-hindering-deserialization-attacks-via-static-duck-typing-44934

Herkunft des Begriffes „Meme“

Der Begriff „Meme“ bezeichnet kurze Videos und Bilder, die mit Schriftzügen, Bildmanipulation oder Musik immer neu interpretiert oder abgeändert werden, um witzige Aussagen zu treffen (Quelle: Duden). Dafür werden oft Abbildungen von berühmten Persönlichkeiten, klassischer Kunst, Comics, Stock Fotos und weiteren originalen Motiven so bearbeitet, dass ein neuer Inhalt entsteht.

Das Wort „Meme“ wurden vom griechischen Wort „mīmēma“ abgewandelt und bedeutet so viel wie „etwas Nachgeahmtes“. Ursprünglich vermutlich in der Genetik durch den Evolutionsbiologen Richard Dawkins in seinem Buch „Das egoistische Gen“ etabliert, verbreitete sich der Begriff später auch in der Internetkultur.

Ähnlich wie ein Gen soll sich laut Dawkins und anderen Wissenschaftlern ein Meme, das für eine Informationseinheit steht, durch Wiederholung weiterverbreiten. Dabei kann es sich auch um einen Witz handeln, den einzelne Personen besonders lustig finden, und den sie deshalb weitererzählen, sodass er zum Meme wird (Quelle: NYT).

Ein bekanntes Beispiel für ein Internet-Meme ist das Doge-Meme. Aus dem Bild eines knuffigen Shiba-Inus (so heißt die Hunderasse) wurde die Grundlage für ein ganzes Meme-Format, aus dem eine Reihe lustige Bilder entstanden sind. Irgendwann wurde daraus sogar eine eigene Krypto-Währung namens Dogecoin.

Wo findet man Memes?

Memes sind eine eigene Subkultur, die ihr auf allen populären Sozialen Medien des Internets findet. Über Social-Media-Kanäle auf Tumblr, Facebook, Twitter oder Instagram werden täglich Millionen Memes verschickt und gepostet. Auf Plattformen wie 9gag und Reddit sind Memes aber besonders häufig zu sehen.

Es gibt auch Memes, die sich auf das Leben in Deutschland beziehen.

Arten von Memes

Es gibt viele Arten von Memes. Am besten lassen sie sich nach ihrem thematischem Inhalt unterteilen. Allerdings sei hier gesagt, dass es zu viele Meme-Kategorien gibt, um sie alle aufzulisten. Im Folgenden bekommt ihr aber einen Überblick über die wichtigsten Arten.

Diese Memes haben meist einen ernsteren Charakter und befassen sich mit einem aktuellen Thema. Dabei bekommen oft bekannte Politiker oder Parteien „ihr Fett weg“. Ein Beispiel dafür wäre ein Meme zu Angela Merkel und ihrer Aussage: „Das Internet ist für uns alle Neuland.“

Solche Bilder und Clips werden besonders zu Videospielen oder Filmen erstellt. Besonders beliebt sind hier die Filmreihen Star Wars oder Herr der Ringe. Aber auch Videospiele fanden blitzschnell Einzug in die Meme-Kultur. Auch Sport-Memes stehen bei vielen auf der Tagesordnung. Besonders, wenn man konkurrierende Vereine ärgern kann.   

Diese Memes sind besonders beliebt, da sie mit den Werken von musikalischen Idolen ihren Schabernack treiben. Beliebt sind in dieser Kategorie zum Beispiel Parodie-Songs. Auch sehr beliebt sind Videos von tanzenden Tieren.

Eine Ikone des Internets ist das Meme „Bongo Cat“ – eine Cartoon-Katze, die auf verschiedenen Instrumenten bekannte Lieder nachspielt, wie zum Beispiel „Never Gonna Give You Up“ von Rick Astley – ein Song, der selbst schon längst zum Meme geworden ist.

Diese Art von Meme lässt sich schon nicht mehr so leicht erklären. Ursprünglich bedeutet das englische Wort „dank“ so viel wie „nasskalt“ oder „feucht“. Im Internet-Slang meint ein Dank Meme jedoch Inhalte, die besonders seltsam sein sollen und von seiner originalen Vorlage recht weit abgewandelt wurden.

Man erkennt Dank Memes an besonders starker Bildbearbeitung, beispielsweise durch Verzerren oder Verpixeln. Videos mit Dank-Memes sind oft zusätzlich mit völlig übersteuerter Musik unterlegt. Lustig sind sie vor allem für diejenigen, die die enthaltenen Insider verstehen oder ihres Memes gerne so absurd wie möglich haben wollen.

Memes selber erstellen und posten

Um selbst Memes zu gestalten, braucht ihr genau zwei Dinge: ein simples Bild- oder Video-Bearbeitungsprogamm und eine Idee. Ihr braucht dafür keine kostenpflichtige Premium-Software. Für kostenlose Bildbearbeitung empfehlen wir GIMP. Aber auch Canva oder der Meme-Generator von Adobe Express sind beliebte Optionen. Kostenlose Videobearbeitung funktioniert beispielsweise mit Shotcut.

Memes und Urheberrecht: Darauf solltet ihr achten

Wichtig bei der Frage nach einer möglichen Urheberrechtsverletzung ist, ob es sich bei eurem Meme um eine persönliche geistige Schöpfung handelt. Je nach Meme muss daher im Einzelfall geprüft werden, ob das Werk individuell und kreativ genug ist, um als eigenständige Neuschöpfung durchzugehen.

Generell fallen Memes üblicherweise in die Kategorie der Parodie, Karikatur oder Pastiche und sind damit zulässig. Vorsicht ist jedoch geboten, wenn das Recht auf das eigene Bild verletzt werden könnte oder wenn Unternehmen gegen Markenrechte verstoßen (Quelle: Urheberrecht.de).

Often, when making decisions based on data, we feel as if we have made a more intelligent, accurate choice. The reality is a little different. From using it to purchase property, to deciding to go on a diet, to selecting a new board member for your company, data can be an amazing catalyst to help make the worst decisions.

Using real-world examples, this article covers common ways data can drive us over a cliff of misinformation. Fortunately, this article also gives actionable, easy-to-use advice on how to avoid such scenarios by following a four-step ladder to assess the quality of data-driven insights:

1. evaluating the data source,
2. considering the bias of the data presenter,
3. recognising the bias of the data reader,
4. identifying any logical missteps between the data and the insight.

TL;DR

• Always question the source of the data. If the source isn’t credible or isn’t provided, don’t be shy in rejecting the entire analysis.
• Don’t hesitate to fact-check claims, especially when they seem counterintuitive.
• Be aware of the data presenters bias. Ask yourself if they’d present the same analysis if it led to the opposite conclusion.
• Be aware of confirmation bias. Ask yourself if you’d accept the same analysis if it led to the opposite conclusion.
• Consider whether the analysis makes logical sense. Just because two things are correlated doesn’t mean one causes the other.
• Recognise that data interpretation often involves assumptions and leaps in logic. It’s crucial to identify and evaluate these leaps using your own judgment.

Question One — Where did the data come from?

Here is an excerpt from an article published in The Spectator, titled: ‘The unfashionable truth about the riots’. Its subject is the 2024 anti-immigration riots in the UK.

…I decided to do some checking on the employment stats for some of the northern towns that have seen the worst rioting in the past week. I also checked the 2011 statistics and then compared the two. I should warn you in advance that if you’re easily depressed, you should look away now.

Back in 2011, the proportion on out-of-work benefits (including incapacity benefit) in Sunderland was 18 per cent; today it is 19 per cent. In 2011 the unemployment figure in Rotherham was 16 per cent; today it is 18 per cent. In Hartlepool, it was 21 per cent; today, 23 per cent.

— The unfashionable truth about the riots, The Spectator

So, where did this data come from?

In the example, all we have is “the employment stats”… What the heck does that mean? In the UK there isn’t a centralised place for employment statistics, so it’s far from obvious what data the author is referring to. You might be content with dismissing the data and thus dismissing the rest of the article, since its premise is based on this data. However, I’m a try-hard, so I put on my best Sherlock Holmes hat and set about finding the un-cited data.

Sadly, I am not Sherlock Holmes. Despite considerable online sleuthing, I couldn’t find any data that matched the article’s. I did, however, find other “employment stats”. Specifically the Office of National Statistics (ONS)’s measure of unemployment by region. It paints a very different picture. In fact the polar opposite picture! Of the three areas the article highlights as having reduced in employment since 2011, the ONS in fact finds they all increased!* I’m afraid the only unfashionable truth illuminated by this article is the degradation of news media. (Or was it always like this?)

*Data links: Sunderland, Rotherham, and Hartlepool employment stats.

Question Two — What if the opposite result were true?

…Using laboratory and field experiments, we find that signing before–rather than after–the opportunity to cheat makes ethics salient when they are needed most and significantly reduces dishonesty.

— Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end, Lisa L. Shu et al.

This quote comes from the abstract of a highly influential research paper in Behavioural Science. In case you didn’t quite understand the quote, since it is written in the classic academic obfuscation style, the quote is saying that signing your name on the top of a document makes you less likely to lie about the document than if you had signed on the bottom.

The data came from a collection of researchers based in highly regarded universities, most notably Francesca Gino who was a professor at Harvard Business School at the time of publication. For me, this passes our first question about the data (‘Where did the data come from?’).

Next we need to ask: ‘Would the data be presented if the opposite result were true?’ To answer this we first need to understand a little about the people and/or institutes presenting the data. In this case it is a group of Behavioural Science academics. Academia is a cut-throat industry, with an exceptionally high failure rate. Therefore, it is critical for academics to be considered ‘successful’ as early as possible to avoid being culled. The key factor to determine an academic’s success is, unsurprisingly, their work. But how do you measure the quality of their work? A common method is citation count, that is the number of times an academic’s work is mentioned in the work of other academics.

The issue with this is you are far more likely to to be cited if you have an unexpected/interesting result in your paper. Going back to the original quote, do you think a result where there was no difference in honesty between the group who signed at the top vs the bottom of a document would have been nearly as influential? No, because that would have been the expectation. The conclusion is that there is a strong incentive for academics to generate unexpected/interesting results. Therefore, in our example, we should understand that there is a strong bias for the authors to claim a (significant) difference in honesty between the two groups.

This sort of bias is so strong that it has led the majority of scientific research to be unreproducible by fellow academics. Unreproducible means an academic writes a research paper that copies another academic’s research paper’s method/experiment, but ends up with significantly different results to the original paper. Turning once more to our case study as an example, following the impactful success of the research paper, a number of other studies attempted to run the same or similar experiment. All results of the experiments found no correlation between honesty and location of document signature. It turned out the original paper’s finding was very likely the result of data fraud. As were a number of other influential papers co-authored by the prestigious Harvard professor, Francesca Gino.

This sort of skin-in-the-game bias isn’t only found in academia. From tech vendors posting their latest performances vs competitors, to investment banks presenting to prospective clients that they are no. 1 in the league tables*, bias is rife. And it isn’t only other people you should be concerned about. The strongest bias of all is most likely you—yes, YOU. Commonly referred to as confirmation bias, it is the tendency to search for, interpret, favour, and recall information in a way that confirms or supports your prior beliefs.

Ever wondered why such a mass of people could be so stupid on so many political topics? Sadly, it is more likely a result of your own confirmation bias than the collective dim-wittedness of millions of people.

If you want to avoid the pitfalls of your own biases, as well as the biases of the data presenter, you not only need to ask: ‘Would the data be presented if the opposite result were true?’ But also: ‘Would I consider the data presented if the opposite result were true?’ It may be a tough pill to swallow, but overcoming your own bias may enlighten you more than all the research in the world.

*May Contain Lies, Alex Edmans. Page 130.

Question Three — Is there a flaw with the leap from data to insight?

For most university graduates, having a degree pays.

Over the course of a lifetime, estimates suggest women can expect to earn about £250,000 more if they have a degree, while the figure is roughly £170,000 for men.

— The degrees that make you rich… and the ones that don’t, BBC News

This quote comes from the news department of the prestigious British Broadcasting Corporation (BBC). Specifically, their article titled: ‘The degrees that make you rich… and the ones that don’t’. First things first, is there a reliable data source?

Great news, the answer is (probably*) yes! The article cites their data from another prestigious organisation, the Institute of Fiscal Studies. You may want to delve deeper into the exact report of the data source, but for a surface-level analysis, I’m happy with where this raw data has come from.

Second we want to ask ourselves, ‘would this information be presented if the opposite result was true?’ Given the BBC has (to my knowledge) no reason to support higher education institutions, it’s fair to assume that the BBC would have published an article highlighting no financial benefits to studying at university if the data suggested it.

However, unfortunately, we still can’t trust the BBC’s findings just yet. We have one final important question to ask: ‘Does the analysis make sense?’ After looking at the article’s chart on average earnings by subject, have a go at answering the following question before moving on to the next paragraph: ‘Given people with Medicine degrees earn more than the average degree, does it make sense to assume that the Medicine degree was the cause of the higher salary?’

My answer: Medicine degrees are necessary to become qualified doctors, and doctors generally earn a high income. This supports the idea that a Medicine degree is the specific reason for average higher future earnings. On the other hand, you don’t just need to pick Medicine, you also need to be picked for Medicine. Given the average A-level results for a UK Medicine graduate is an outstanding AAA, you could argue the reason for the high average salaries is because the average Medicine student is exceptionally cognitively capable (in academia at least), which has nothing to do with whether they actually chose Medicine as a degree or not.

What about the degree with the second highest average salary, Economics? Unlike Medicine, you don’t strictly need an Economics degree to qualify for any specific high-paying jobs. I’d argue an Economics degree is much less likely a direct factor in increasing a graduate’s future salary. For example, someone who is genuinely interested in money at school will be more likely to base their career decisions on what pays more. That same person may also be more likely to pick Economics at university.** If that’s true, then pushing someone who is not interested in money to study Economics may not benefit their future career or financial prospects as much as the article is implying.

Similarly, someone who earns a university degree in general may be little to no better off than if they hadn’t gone to university at all. It may simply be a side effect. For example, in the UK children with wealthy parents are significantly more likely to attend university. The wealth of their parents may also have an impact on the child’s high-paying job prospects; for example, they might provide invaluable knowledge to their children on how to interview, climb the greasy corporate ladder, and/or network. Given this thought process, the article’s finding that “For most university graduates, having a degree pays” is, at the very least, not a proven fact, and should not be treated as such.

Deciding what ‘makes sense’ is subjective; for example, you may not have been convinced by my arguments above. However, it’s important to recognise the leaps in assumptions people make from data to insights, and to do your own ‘makes sense’ assessment before accepting them.

*It’s important to acknowledge the limitations of time we have on the depth of our investigation. For very serious things you will want to go deeper, but as a pragmatist, it isn’t possible for every data analysis we see.

**You may be thinking: ‘hang on a minute, you didn’t provide any data to back up that theory’. And you’re right! However, often times we don’t have all the data available to us, we simply have to use our own experiences and intuition. As an Economics graduate who has also been fascinated with finance from an early age, the theory makes sense to me.

Wrapping up

In conclusion, data should not be the driver of your decisions. Like an unpredictable friend, data belongs firmly in the passenger seat, preferably with the child-lock on. They can provide suggestions, but ultimately you need to be sceptical before making any decision based off of them.

Remember, when looking into data, statistics, and/or research, you need to ask yourself:

1. Where did this data come from?
2. Would this data be presented if the opposite result were true?
3. Would I consider the data if the opposite result were true?
4. Is there a flaw with the leap from data to insight?

Final note for data scientists/analysts

For data scientists and analysts, these questions are not just a safeguard — they’re a responsibility. As gatekeepers of data-driven insights, your role is crucial in ensuring that decisions are informed by sound reasoning, not just raw numbers and algorithms. By applying this mindset, you can avoid being that unpredictable friend in your organisation. And who know’s, one day you may even be invited into the driving seat.

Stop Being Data-Driven was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Often, when making decisions based on data, we feel as if we have made a more intelligent, accurate choice. The reality is a little different. From using it to purchase property, to deciding to go on a diet, to selecting a new board member for your company, data can be an amazing catalyst to help make the worst decisions.

Using real-world examples, this article covers common ways data can drive us over a cliff of misinformation. Fortunately, this article also gives actionable, easy-to-use advice on how to avoid such scenarios by following a four-step ladder to assess the quality of data-driven insights:

1. evaluating the data source,
2. considering the bias of the data presenter,
3. recognising the bias of the data reader,
4. identifying any logical missteps between the data and the insight.

TL;DR

• Always question the source of the data. If the source isn’t credible or isn’t provided, don’t be shy in rejecting the entire analysis.
• Don’t hesitate to fact-check claims, especially when they seem counterintuitive.
• Be aware of the data presenters bias. Ask yourself if they’d present the same analysis if it led to the opposite conclusion.
• Be aware of confirmation bias. Ask yourself if you’d accept the same analysis if it led to the opposite conclusion.
• Consider whether the analysis makes logical sense. Just because two things are correlated doesn’t mean one causes the other.
• Recognise that data interpretation often involves assumptions and leaps in logic. It’s crucial to identify and evaluate these leaps using your own judgment.

Question One — Where did the data come from?

Here is an excerpt from an article published in The Spectator, titled: ‘The unfashionable truth about the riots’. Its subject is the 2024 anti-immigration riots in the UK.

…I decided to do some checking on the employment stats for some of the northern towns that have seen the worst rioting in the past week. I also checked the 2011 statistics and then compared the two. I should warn you in advance that if you’re easily depressed, you should look away now.

Back in 2011, the proportion on out-of-work benefits (including incapacity benefit) in Sunderland was 18 per cent; today it is 19 per cent. In 2011 the unemployment figure in Rotherham was 16 per cent; today it is 18 per cent. In Hartlepool, it was 21 per cent; today, 23 per cent.

— The unfashionable truth about the riots, The Spectator

So, where did this data come from?

In the example, all we have is “the employment stats”… What the heck does that mean? In the UK there isn’t a centralised place for employment statistics, so it’s far from obvious what data the author is referring to. You might be content with dismissing the data and thus dismissing the rest of the article, since its premise is based on this data. However, I’m a try-hard, so I put on my best Sherlock Holmes hat and set about finding the un-cited data.

Sadly, I am not Sherlock Holmes. Despite considerable online sleuthing, I couldn’t find any data that matched the article’s. I did, however, find other “employment stats”. Specifically the Office of National Statistics (ONS)’s measure of unemployment by region. It paints a very different picture. In fact the polar opposite picture! Of the three areas the article highlights as having reduced in employment since 2011, the ONS in fact finds they all increased!* I’m afraid the only unfashionable truth illuminated by this article is the degradation of news media. (Or was it always like this?)

*Data links: Sunderland, Rotherham, and Hartlepool employment stats.

Question Two — What if the opposite result were true?

…Using laboratory and field experiments, we find that signing before–rather than after–the opportunity to cheat makes ethics salient when they are needed most and significantly reduces dishonesty.

— Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end, Lisa L. Shu et al.

This quote comes from the abstract of a highly influential research paper in Behavioural Science. In case you didn’t quite understand the quote, since it is written in the classic academic obfuscation style, the quote is saying that signing your name on the top of a document makes you less likely to lie about the document than if you had signed on the bottom.

The data came from a collection of researchers based in highly regarded universities, most notably Francesca Gino who was a professor at Harvard Business School at the time of publication. For me, this passes our first question about the data (‘Where did the data come from?’).

Next we need to ask: ‘Would the data be presented if the opposite result were true?’ To answer this we first need to understand a little about the people and/or institutes presenting the data. In this case it is a group of Behavioural Science academics. Academia is a cut-throat industry, with an exceptionally high failure rate. Therefore, it is critical for academics to be considered ‘successful’ as early as possible to avoid being culled. The key factor to determine an academic’s success is, unsurprisingly, their work. But how do you measure the quality of their work? A common method is citation count, that is the number of times an academic’s work is mentioned in the work of other academics.

The issue with this is you are far more likely to to be cited if you have an unexpected/interesting result in your paper. Going back to the original quote, do you think a result where there was no difference in honesty between the group who signed at the top vs the bottom of a document would have been nearly as influential? No, because that would have been the expectation. The conclusion is that there is a strong incentive for academics to generate unexpected/interesting results. Therefore, in our example, we should understand that there is a strong bias for the authors to claim a (significant) difference in honesty between the two groups.

This sort of bias is so strong that it has led the majority of scientific research to be unreproducible by fellow academics. Unreproducible means an academic writes a research paper that copies another academic’s research paper’s method/experiment, but ends up with significantly different results to the original paper. Turning once more to our case study as an example, following the impactful success of the research paper, a number of other studies attempted to run the same or similar experiment. All results of the experiments found no correlation between honesty and location of document signature. It turned out the original paper’s finding was very likely the result of data fraud. As were a number of other influential papers co-authored by the prestigious Harvard professor, Francesca Gino.

This sort of skin-in-the-game bias isn’t only found in academia. From tech vendors posting their latest performances vs competitors, to investment banks presenting to prospective clients that they are no. 1 in the league tables*, bias is rife. And it isn’t only other people you should be concerned about. The strongest bias of all is most likely you—yes, YOU. Commonly referred to as confirmation bias, it is the tendency to search for, interpret, favour, and recall information in a way that confirms or supports your prior beliefs.

Ever wondered why such a mass of people could be so stupid on so many political topics? Sadly, it is more likely a result of your own confirmation bias than the collective dim-wittedness of millions of people.

If you want to avoid the pitfalls of your own biases, as well as the biases of the data presenter, you not only need to ask: ‘Would the data be presented if the opposite result were true?’ But also: ‘Would I consider the data presented if the opposite result were true?’ It may be a tough pill to swallow, but overcoming your own bias may enlighten you more than all the research in the world.

*May Contain Lies, Alex Edmans. Page 130.

Question Three — Is there a flaw with the leap from data to insight?

For most university graduates, having a degree pays.

Over the course of a lifetime, estimates suggest women can expect to earn about £250,000 more if they have a degree, while the figure is roughly £170,000 for men.

— The degrees that make you rich… and the ones that don’t, BBC News

This quote comes from the news department of the prestigious British Broadcasting Corporation (BBC). Specifically, their article titled: ‘The degrees that make you rich… and the ones that don’t’. First things first, is there a reliable data source?

Great news, the answer is (probably*) yes! The article cites their data from another prestigious organisation, the Institute of Fiscal Studies. You may want to delve deeper into the exact report of the data source, but for a surface-level analysis, I’m happy with where this raw data has come from.

Second we want to ask ourselves, ‘would this information be presented if the opposite result was true?’ Given the BBC has (to my knowledge) no reason to support higher education institutions, it’s fair to assume that the BBC would have published an article highlighting no financial benefits to studying at university if the data suggested it.

However, unfortunately, we still can’t trust the BBC’s findings just yet. We have one final important question to ask: ‘Does the analysis make sense?’ After looking at the article’s chart on average earnings by subject, have a go at answering the following question before moving on to the next paragraph: ‘Given people with Medicine degrees earn more than the average degree, does it make sense to assume that the Medicine degree was the cause of the higher salary?’

My answer: Medicine degrees are necessary to become qualified doctors, and doctors generally earn a high income. This supports the idea that a Medicine degree is the specific reason for average higher future earnings. On the other hand, you don’t just need to pick Medicine, you also need to be picked for Medicine. Given the average A-level results for a UK Medicine graduate is an outstanding AAA, you could argue the reason for the high average salaries is because the average Medicine student is exceptionally cognitively capable (in academia at least), which has nothing to do with whether they actually chose Medicine as a degree or not.

What about the degree with the second highest average salary, Economics? Unlike Medicine, you don’t strictly need an Economics degree to qualify for any specific high-paying jobs. I’d argue an Economics degree is much less likely a direct factor in increasing a graduate’s future salary. For example, someone who is genuinely interested in money at school will be more likely to base their career decisions on what pays more. That same person may also be more likely to pick Economics at university.** If that’s true, then pushing someone who is not interested in money to study Economics may not benefit their future career or financial prospects as much as the article is implying.

Similarly, someone who earns a university degree in general may be little to no better off than if they hadn’t gone to university at all. It may simply be a side effect. For example, in the UK children with wealthy parents are significantly more likely to attend university. The wealth of their parents may also have an impact on the child’s high-paying job prospects; for example, they might provide invaluable knowledge to their children on how to interview, climb the greasy corporate ladder, and/or network. Given this thought process, the article’s finding that “For most university graduates, having a degree pays” is, at the very least, not a proven fact, and should not be treated as such.

Deciding what ‘makes sense’ is subjective; for example, you may not have been convinced by my arguments above. However, it’s important to recognise the leaps in assumptions people make from data to insights, and to do your own ‘makes sense’ assessment before accepting them.

*It’s important to acknowledge the limitations of time we have on the depth of our investigation. For very serious things you will want to go deeper, but as a pragmatist, it isn’t possible for every data analysis we see.

**You may be thinking: ‘hang on a minute, you didn’t provide any data to back up that theory’. And you’re right! However, often times we don’t have all the data available to us, we simply have to use our own experiences and intuition. As an Economics graduate who has also been fascinated with finance from an early age, the theory makes sense to me.

Wrapping up

In conclusion, data should not be the driver of your decisions. Like an unpredictable friend, data belongs firmly in the passenger seat, preferably with the child-lock on. They can provide suggestions, but ultimately you need to be sceptical before making any decision based off of them.

Remember, when looking into data, statistics, and/or research, you need to ask yourself:

1. Where did this data come from?
2. Would this data be presented if the opposite result were true?
3. Would I consider the data if the opposite result were true?
4. Is there a flaw with the leap from data to insight?

Final note for data scientists/analysts

For data scientists and analysts, these questions are not just a safeguard — they’re a responsibility. As gatekeepers of data-driven insights, your role is crucial in ensuring that decisions are informed by sound reasoning, not just raw numbers and algorithms. By applying this mindset, you can avoid being that unpredictable friend in your organisation. And who know’s, one day you may even be invited into the driving seat.

Stop Being Data-Driven was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Read more of this story at Slashdot.

C2 solution that communicates directly over Bluetooth-Low-Energy with your Bash Bunny Mark II.
Send your Bash Bunny all the instructions it needs just over the air.

Overview

Structure

Installation & Start

1. Install required dependencies

pip install pygatt "pygatt[GATTTOOL]"

Make sure BlueZ is installed and gatttool is usable

sudo apt install bluez

2. Download BlueBunny's repository (and switch into the correct folder)

git clone https://github.com/90N45-d3v/BlueBunny
cd BlueBunny/C2

3. Start the C2 server

sudo python c2-server.py

4. Plug your Bash Bunny with the BlueBunny payload into the target machine (payload at: BlueBunny/payload.txt).
5. Visit your C2 server from your browser on localhost:1472 and connect your Bash Bunny (Your Bash Bunny will light up green when it's ready to pair).

Manual communication with the Bash Bunny through Python

You can use BlueBunny's BLE backend and communicate with your Bash Bunny manually.

Example Code

# Import the backend (BlueBunny/C2/BunnyLE.py)
import BunnyLE

# Define the data to send
data = "QUACK STRING I love my Bash Bunny"
# Define the type of the data to send ("cmd" or "payload") (payload data will be temporary written to a file, to execute multiple commands like in a payload script file)
d_type = "cmd"

# Initialize BunnyLE
BunnyLE.init()

# Connect to your Bash Bunny
bb = BunnyLE.connect()

# Send the data and let it execute
BunnyLE.send(bb, data, d_type)

Troubleshooting

Connecting your Bash Bunny doesn't work? Try the following instructions:

• Try connecting a few more times
• Check if your bluetooth adapter is available
• Restart the system your C2 server is running on
• Check if your Bash Bunny is running the BlueBunny payload properly
• How far away from your Bash Bunny are you? Is the environment (distance, interferences etc.) still sustainable for typical BLE connections?

Bugs within BlueZ

The Bluetooth stack used is well known, but also very buggy. If starting the connection with your Bash Bunny does not work, it is probably a temporary problem due to BlueZ. Here are some kind of errors that can be caused by temporary bugs. These usually disappear at the latest after rebooting the C2's operating system, so don't be surprised and calm down if they show up.

• Timeout after 5.0 seconds
• Unknown error while scanning for BLE devices

Working on...

• Remote shell access
• BLE exfiltration channel
• Improved connecting process

Additional information

As I said, BlueZ, the base for the bluetooth part used in BlueBunny, is somewhat bug prone. If you encounter any non-temporary bugs when connecting to Bash Bunny as well as any other bugs/difficulties in the whole BlueBunny project, you are always welcome to contact me. Be it a problem, an idea/solution or just a nice feedback.

In this post we will show how to build a simple end-to-end application in the cloud on a serverless infrastructure. The purpose is simple: we want to show that we can develop directly against the cloud while minimizing the cognitive overhead of designing and building infrastructure. Plus, we will put together a design that minimizes costs compared to modern data warehouses, such as Big Query or Snowflake.

As data practitioners we want (and love) to build applications on top of our data as seamlessly as possible. Whether you work in BI, Data Science or ML all that matters is the final application and how fast you can see it working end-to-end. The infrastructure often gets in the way though.

Imagine, as a practical example, that we need to build a new customer-facing analytics application for our product team. Because it’s client-facing we have performance constraints we need to respect, such as low latency.

We can start developing it directly in the cloud, but it would immediately bring us to some infra questions: where do we run it? How big a machine do we need? Because of the low-latency requirement, do we need to build a caching layer? If so, how do we do it?

Alternatively, we can develop our app locally. It will probably be more intuitive from the developer experience point of view but it only postpones the infra questions, since in the end we will have to find a way to go from our local project to actual pipelines. Moreover, the data will need to leave the cloud env to go on our machine, which is not exactly secure and auditable.

To make the cloud experience as smooth as possible we designed a data lake architecture where data are sitting in a simple cloud storage (AWS S3) and a serverless infrastructure that embeds DuckDB works as a query engine. At the end of the cycle, we will have an analytics app that can be used to both visualize and query the data in real time with virtually no infra costs.

Of course, this is a bit of a simplification, as some tweaks would be needed to run this project in real production scenarios. What we provide is a general blueprint to leverage the separation of storage and compute to build a data lake with a query engine in the cloud. We show how to power an interactive data app with an (almost) free cloud endpoint, no warehouse setup, and lighting fast performance. In our implementation, the final application is a simple Streamlit app, but that’s merely for explanatory purposes: you can easily think of plugging in your favorite BI tool.

Ducks go serverless

Y’all know DuckDB at this point. It is an open-source in-process SQL OLAP database built specifically for analytical queries. It is somewhat still unclear how much DuckDB is actually used in production, but for us today the killer feature is the possibility of querying parquet files directly in S3 with SQL syntax.

So most practitioners seem to be using it right now as a local engine for data exploration, ad hoc analysis, POCs and prototyping (with some creative ideas on how to extend its initial purpose to cover more surface:). People create notebooks or small data apps with embedded DuckDB to prototype and experiment with production data locally.

fs111 @fs111@mastodon.xyz on Twitter: "It is infinitely nicer to use @duckdb to quickly look at @ApacheParquet files than using any of the horrible hadoop/spark things. / Twitter"

It is infinitely nicer to use @duckdb to quickly look at @ApacheParquet files than using any of the horrible hadoop/spark things.

The cloud is better. And if we often feel it isn’t, it’s because something is wrong with the tool chain we use, but there is a very big difference between a bad idea and a good idea badly executed.

If we combine a data lake architecture, a serverless design, DuckDB and a bit of ingenuity we can build a very fast data stack from spare parts: no warehouse setup, lighting fast performance and outrageously cheap costs — S3 most expensive standard pricing is $0.023 per GB, AWS Lambda is very fast and scales to zero when not in use, so it is a no-fat computation bill, plus AWS gives you 1M calls for free.

Buckle up, clone the repo, sing along and for more details, please refer to the README.

Architecture

This project is pretty self-contained and requires only introductory-level familiarity with cloud services and Python.

The idea is to start from a Data Lake where our data are stored. Once the data is uploaded in our S3 in a parquet format, we can then trigger the lambda with a SQL query. At that point, our lambda goes up, spins up a DuckDB instance in memory, computes the query and gets back to the user with the results, which can be directly rendered as tables in the Terminal.

The architecture has a number of advantages mostly coming from the serverless design: speed, proximity to the data and one line deployment are nice; plus, of course, the system scales to zero when not used, so we only pay per query.

Your first query engine + data lake from spare parts

We provide a simple script that will create an S3 bucket and populate it with a portion of the NYC TLC Trip Record Data (available under the nyc.gov terms of use), both as a unique file and as a hive-partitioned directory (you can run it with Make). Once the data are in the data lake we can set up and use our lambda: if you have the Serverless CLI setup correctly, deploying the lambda is one command of Make again.

The lambda can be invoked in any of the usual ways, and accepts a query as its main payload: when it runs, it uses DuckDB (re-using an instance if on a warm start) to execute the query on the data lake. DuckDb does not know anything about the data before and after the execution, making the lambda purely stateless as far as data semantics is concerned.

For instance, you can use the simple Python script in the project to send this query to the lambda, and display the results:

SELECT
  pickup_at as pickup_time,
  dropoff_at as dropoff_location,
  trip_distance
FROM read_parquet(your_s3_bucket/dataset/taxi_2019_04.parquet')
WHERE pickup_at >= '2019–04–01' AND pickup_at < '2019–04–03'
LIMIT 10

Et voilà! You can now query your data lake, securely in the cloud. This very simple design addresses directly two of the typical frictions for working in cloud data warehouses:

• The setup is significantly simplified since all the user needs to do is to have her AWS credential. Once the setup is done, the user only needs access to the lambda (or any proxy to it!): that is good, as it gives the user full query capabilities without access to the underlying storage.
• The performances are so good that it feels like developing locally, even if we always go through the cloud. A snappy cloud experience helps tame the too familiar feeling that advantages of working on remote machines is paid in the coin of good developer experience.

(Almost) Free Analytics

It’s all good and boujee, but let us say that we want to do a bit more than query data on the fly. Let’s say that we want to build an application on top of a table. It’s a very simple app, there is no orchestration and no need to calibrate the workload.

At the same time, let’s say that this application needs to be responsive, it needs to be fast. Anyone who deals with clients directly, for instance, knows how it is important to provide a fresh responsive experience to the clients who want to see their data. The one thing nobody likes is a dashboard that takes minutes to load.

To see how this architecture can bridge the gap between data pipelines and real-time querying for analytics, we provide a small dbt DAG to simulate running some offline SQL transformations over the original dataset resulting in a new artifact in the data lake (the equivalent of a dashboard view).

To keep things as self contained as possible, we included a version that you can run locally on your machine (see README for more details — nor dbt nor the engine behind it matter for this pattern to work). However, you can use a different runtime for dbt and export the final artifact as a parquet file, with Snowflake or BigQuery.

For the time being, we’ll stick to our super simple DAG made of two nodes. The first node takes the pickup_location_id from our data lake and order them by the number of trips:

SELECT
  pickup_location_id AS location_id,
COUNT(*) AS counts
FROM read_parquet(['s3://{{ env_var('S3_BUCKET_NAME') }}/dataset/taxi_2019_04.parquet'])
GROUP BY 1

The second that gives as the top 200 pick up locations in our data set:

SELECT
  location_id,
  counts
FROM {{ ref('trips_by_pickup_location') }}
ORDER BY 2 DESC
LIMIT 200

We can visualize the DAG with dbt docs:

Once our pipeline is done, the final artifact is uploaded back to our data lake in:

s3:/your_s3_bucket/dashboard/my_view.parquet

We can then reuse the query engine we built before to query the second (and final) node of our DAG to visualize the data in a Streamlit app, simply by running in the terminal:

(venv) cirogreco@Ciros-MBP src % make dashboard

Every time we hit the dashboard, the dashboard hits the lambda behind the scenes. If you like this simple architecture, the same pattern can be used in your own Streamlit app, or in your favorite BI tool.

A few remarks on the “Reasonable Scale”

A while ago, we wrote a series of posts on what we called MLOps at Reasonable Scale where we talked about the best strategies to build reliable ML applications in companies that do not process data at internet scale and have a number of constraints that truly Big-Data companies typically do not have. We mostly talked about it from the point of view of ML and MLOps because operationalizing successfully ML was a major problem for organizations at the time (maybe it still is, I am not sure), but one general observation remains: most data organizations are “Reasonable Scale” and they should design their systems around this assumption. Note that being a reasonable scale organization does not necessarily mean being a small company. The enterprise world is full of data teams who deal with a lot of complexity within large — sometimes enormous — organizations and yet have many reasonable scale pipelines, often for internal stakeholders, ranging from a few GB to at most a TB.

Recently, we happily witnessed a growing debate around whether companies need Big Data systems to deal with their data problems,. The most important takeaway from our point of view remains that, if you are a Reasonable Scale organization, dealing with unnecessary infrastructure can be a very serious burden with plenty of nefarious ramifications in your organization processes. You could in principle build an entire data stack to support low latency dashboards — maybe you could use a Data Warehouse and a caching layer -, but since your resources are limited, wouldn’t it be nice to have a simpler and cheaper way?

In this post, we showed that the combination of data-first storage formats, on-demand compute and in-memory OLAP processing opens up for new possibilities at Reasonable Scale. The system is far from perfect and could use many improvements, but it shows that one can build an interactive data app with no warehouse setup, lighting fast performances and virtually no costs. By removing the db from DuckDB, we can combine what is fundamentally right about local (“single node processing is all you need”) with what is fundamentally right about the cloud (“data is better processed elsewhere”).

A Serverless Query Engine from Spare Parts was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

Die neue Ducky-Tastatur One 3 SF setzt auf „Quack Mechanics“. Das klingt nach Marketing-Geschnatter, hat aber ordentlich Biss. Das neue Quaken macht die One 3 SF zu einem deutlichen Upgrade. Nicht alle Neuerungen sind allerdings eindeutige Fortschritte.

Read more of this story at Slashdot.

From 2007 through 2015, the US Transportation Security Administration (TSA) spent $1.5 billion trying to identify potentially dangerous travelers by observing their behavior through an ongoing program called SPOT.…