Reverse-engineering YouTube Recommenders?

  • Context: Graduate 
  • Thread starter Thread starter WWGD
  • Start date Start date
  • Tags Tags
    Youtube
Click For Summary

Discussion Overview

The discussion revolves around the mechanisms of YouTube's song recommendation system, particularly the idea of reverse-engineering it based on user listening paths. Participants explore the types of algorithms that may influence recommendations and share personal experiences with the system.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested

Main Points Raised

  • One participant suggests using the "path" generated by a specific song as a seed to reverse-engineer the recommender system, expressing uncertainty about the "Levels" issue.
  • Another participant proposes that there are multiple algorithms at work, including one based on individual listening history and another based on associations with what others listen to after a particular song.
  • A later reply reiterates the idea of collaborative filtering and content-based filtering as methods for recommendations, noting that the same initial songs can lead to different paths in recommendations.
  • Links to open-access works related to recommender systems are shared, highlighting contributions from John T. Riedl and discussing historical context and personal anecdotes related to AI and natural language processing.

Areas of Agreement / Disagreement

Participants express multiple competing views regarding the algorithms behind YouTube's recommendations, and the discussion remains unresolved with no consensus on the exact mechanisms at play.

Contextual Notes

There are limitations in understanding the specific algorithms used by YouTube, as well as the definitions of terms like "Levels" and the criteria for determining song similarity.

WWGD
Science Advisor
Homework Helper
Messages
7,806
Reaction score
13,125
Hi,
This is just out of curiosity. I have been hearing music from YouTube for a while. It seems no matter
where I start I end up in the same two songs, as sort of "Attractors".
Can I use the "path" generated by a specific song as a seed ( first one I listen to in a given internet session) and use the somg-path to reverse-engineer the underlying song recommender? EDIT: I am not sure how the LEvels issue works, so I will take any suggestion/idea.
 
Physics news on Phys.org
My guess is there are 2 (or more) types of algorithms at work. One could be based on your own listening history and try to recommend songs similar to ones which you like. There would need to be some method to decide what is similar and what is not similar. Another could be based on an association - what other people listened to after listening to a particular song. Or perhaps a method similar to Google's pagerank system, to figure which ones have the strongest connections.
 
  • Like
Likes   Reactions: WWGD
The following links are to some open-access works co-authored by John T. Riedl, who was an IEEE Fellow, a University of Minnesota Professor, and a great man who had a special knack and compassionate motivation for imagining how people could co-contribute meaningfully in collaborative efforts. I think his sensibilities in that regard are part and parcel of what led him to be among the pioneers of recommender systems.

https://spectrum.ieee.org/computing/software/deconstructing-recommender-systems
https://www.researchgate.net/publication/215470714_Evaluating_collaborative_filtering_recommender_systems
http://www.ra.ethz.ch/cdstore/www10/papers/pdf/p519.pdf
http://files.grouplens.org/papers/ec00.pdf
https://apps.dtic.mil/dtic/tr/fulltext/u2/a439541.pdf

John was a computing buddy of mine when we were in school (early-to-mid '80s), and he once had me co-present some ideas, mostly his own, for machine processing of natural (human) language.

The professors mostly thought the project ideas were too ambitious (possibly partly because we were both undergrads), but they let us get started on it, looking at ways to recognize grammatical constructs such as parts of speech, how to construct context-based semantic disambiguators based partly on analysis of proximities and frequencies of co-lexemes, syntactical facts, etc..

The faculty members questioned what uses we might find for such systems -- I said that minimally, we could enhance spelling checkers to recognize that "to, too, and two" don't always add up to six (that example of course works better in speech than in writing), or with a bit more effort, we could highlight possibly incorrect usage of "affect" versus "effect". John's response was more general, along the lines of "I don't really have much in mind yet about what the applications might be; I just think it'd be interesting; I'm curious about how far we could get with it just as a general purpose AI tool project."

John had previously asked me to write a command line interface to invoke a full-screen editor he'd written in PL/1, modeled after one he'd previously written in COBOL. In response, I'd written a few hundred lines of CLIST code to faithfully mimic the invocatory syntax and file allocation consequences of the IBM TSO EDIT command (a line-oriented, non-full-screen editor), making his editor more conveniently accessible to everyone.

John not only recognized and appreciated the abilities of others; if you were in his vicinity he would find some way for you to pitch on something he was working on, or he'd contribute helpfully to your work -- https://www.richrelevance.com/blog/2013/08/07/remembering-john-riedl/
 
  • Like
Likes   Reactions: WWGD
scottdave said:
My guess is there are 2 (or more) types of algorithms at work. One could be based on your own listening history and try to recommend songs similar to ones which you like. There would need to be some method to decide what is similar and what is not similar. Another could be based on an association - what other people listened to after listening to a particular song. Or perhaps a method similar to Google's pagerank system, to figure which ones have the strongest connections.
I have heard them called collaborative filtering : when your "similars" ( people known to like songs you like too) and content-based filtering : recommendations based on songs one has listened too. Weird thing is that the same seed of two initial songs lead me through different paths/songs.
 

Similar threads

  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 17 ·
Replies
17
Views
2K
  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 9 ·
Replies
9
Views
3K
Replies
23
Views
3K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 39 ·
2
Replies
39
Views
5K
  • · Replies 14 ·
Replies
14
Views
2K
  • · Replies 4 ·
Replies
4
Views
3K
  • · Replies 3 ·
Replies
3
Views
3K