Graduate Reverse-engineering YouTube Recommenders?

  • Thread starter Thread starter WWGD
  • Start date Start date
  • Tags Tags
    Youtube
Click For Summary
SUMMARY

This discussion centers on reverse-engineering YouTube's song recommendation algorithms. The user proposes utilizing the "path" generated by a specific song as a seed to analyze the underlying recommendation system. They identify two primary algorithm types: collaborative filtering, which recommends songs based on user listening history, and content-based filtering, which suggests songs similar to those already liked. The conversation also references the influence of John T. Riedl in the development of recommender systems and highlights the complexity of song recommendation paths.

PREREQUISITES
  • Understanding of collaborative filtering algorithms
  • Familiarity with content-based filtering techniques
  • Knowledge of recommendation system design principles
  • Basic grasp of graph theory concepts, such as PageRank
NEXT STEPS
  • Research "collaborative filtering algorithms" for deeper insights into user-based recommendations
  • Explore "content-based filtering techniques" to understand similarity measures in recommendations
  • Learn about "PageRank algorithm" and its application in recommendation systems
  • Investigate "YouTube API" for practical experimentation with song recommendation data
USEFUL FOR

Data scientists, machine learning engineers, and anyone interested in the mechanics of recommendation systems, particularly in music streaming platforms like YouTube.

WWGD
Science Advisor
Homework Helper
Messages
7,772
Reaction score
13,003
Hi,
This is just out of curiosity. I have been hearing music from YouTube for a while. It seems no matter
where I start I end up in the same two songs, as sort of "Attractors".
Can I use the "path" generated by a specific song as a seed ( first one I listen to in a given internet session) and use the somg-path to reverse-engineer the underlying song recommender? EDIT: I am not sure how the LEvels issue works, so I will take any suggestion/idea.
 
Physics news on Phys.org
My guess is there are 2 (or more) types of algorithms at work. One could be based on your own listening history and try to recommend songs similar to ones which you like. There would need to be some method to decide what is similar and what is not similar. Another could be based on an association - what other people listened to after listening to a particular song. Or perhaps a method similar to Google's pagerank system, to figure which ones have the strongest connections.
 
  • Like
Likes WWGD
The following links are to some open-access works co-authored by John T. Riedl, who was an IEEE Fellow, a University of Minnesota Professor, and a great man who had a special knack and compassionate motivation for imagining how people could co-contribute meaningfully in collaborative efforts. I think his sensibilities in that regard are part and parcel of what led him to be among the pioneers of recommender systems.

https://spectrum.ieee.org/computing/software/deconstructing-recommender-systems
https://www.researchgate.net/publication/215470714_Evaluating_collaborative_filtering_recommender_systems
http://www.ra.ethz.ch/cdstore/www10/papers/pdf/p519.pdf
http://files.grouplens.org/papers/ec00.pdf
https://apps.dtic.mil/dtic/tr/fulltext/u2/a439541.pdf

John was a computing buddy of mine when we were in school (early-to-mid '80s), and he once had me co-present some ideas, mostly his own, for machine processing of natural (human) language.

The professors mostly thought the project ideas were too ambitious (possibly partly because we were both undergrads), but they let us get started on it, looking at ways to recognize grammatical constructs such as parts of speech, how to construct context-based semantic disambiguators based partly on analysis of proximities and frequencies of co-lexemes, syntactical facts, etc..

The faculty members questioned what uses we might find for such systems -- I said that minimally, we could enhance spelling checkers to recognize that "to, too, and two" don't always add up to six (that example of course works better in speech than in writing), or with a bit more effort, we could highlight possibly incorrect usage of "affect" versus "effect". John's response was more general, along the lines of "I don't really have much in mind yet about what the applications might be; I just think it'd be interesting; I'm curious about how far we could get with it just as a general purpose AI tool project."

John had previously asked me to write a command line interface to invoke a full-screen editor he'd written in PL/1, modeled after one he'd previously written in COBOL. In response, I'd written a few hundred lines of CLIST code to faithfully mimic the invocatory syntax and file allocation consequences of the IBM TSO EDIT command (a line-oriented, non-full-screen editor), making his editor more conveniently accessible to everyone.

John not only recognized and appreciated the abilities of others; if you were in his vicinity he would find some way for you to pitch on something he was working on, or he'd contribute helpfully to your work -- https://www.richrelevance.com/blog/2013/08/07/remembering-john-riedl/
 
  • Like
Likes WWGD
scottdave said:
My guess is there are 2 (or more) types of algorithms at work. One could be based on your own listening history and try to recommend songs similar to ones which you like. There would need to be some method to decide what is similar and what is not similar. Another could be based on an association - what other people listened to after listening to a particular song. Or perhaps a method similar to Google's pagerank system, to figure which ones have the strongest connections.
I have heard them called collaborative filtering : when your "similars" ( people known to like songs you like too) and content-based filtering : recommendations based on songs one has listened too. Weird thing is that the same seed of two initial songs lead me through different paths/songs.
 
If there are an infinite number of natural numbers, and an infinite number of fractions in between any two natural numbers, and an infinite number of fractions in between any two of those fractions, and an infinite number of fractions in between any two of those fractions, and an infinite number of fractions in between any two of those fractions, and... then that must mean that there are not only infinite infinities, but an infinite number of those infinities. and an infinite number of those...

Similar threads

  • · Replies 17 ·
Replies
17
Views
2K
  • · Replies 9 ·
Replies
9
Views
2K
  • · Replies 9 ·
Replies
9
Views
2K
Replies
23
Views
3K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 35 ·
2
Replies
35
Views
2K
  • · Replies 14 ·
Replies
14
Views
1K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K