| Author ___________________ |
Title/TLDR ___________________ |
Arugment _______________________________________________________________________________________________ |
Counter Aguments ____________________________________________________________________________ |
Reference URL ___________________ |
Expecting Uncontrollable Risk? ___________________ |
|---|---|---|---|---|---|
| Jaan Tallin | LLMS are currently "summoned and tame" | - The problem is that with with these experiments uh they are producing uncontrollable Minds uh the yeah that's why I call it someone and tame uh Paradigm of AI - You summoned this mind uh from the Mind space using your data and a lot of compute a lot of money and then you try to tame it using things reinforcement learning from Human feedback | https://youtu.be/Dmh6ciu24v0?t=2993 | Yes | |
| Robin Hanson | LLMs Dev should be required to by "insurance" for potential damages | - I would combine this with some sort of required liabilityinsurance that would even strengthen it such that the bigger the damages and the more they have to make sure somebody some insurance company is set up to pay for it and that would also discourage them in the sense that it would be harder to arrange for that insurance and the insurers would say you know you know show me that you're not going too close to this problematic part and they have problem doing that and that's why they would stay away that's the idea | https://youtu.be/Dmh6ciu24v0?t=2999 | No | |
| Robin Hanson | Future LLMs might be smarter then Human Level, but that does not pose a specific risk | - I'm granting that GPT 5 could be superhuman on many performance characteristics what I'm doubting is that that destroys the world. | https://youtu.be/Dmh6ciu24v0?t=3370 | No | |
| Robin Hanson | If people were doing bad things, then I would look into stopping it | - if people were trying to make the worst case systems and like the management and funding of a particular you know most advanced projects were trying to make the worst case scenarios they went out of their way to make those scenarios well then I get a lot more worried about those | - Wow. - People are trying to use GPTs to be bad things already - They wouldn't "broadcast" it if they were, as it would undoughtedly be illegal | https://youtu.be/Dmh6ciu24v0?t=3483 | No |
| Jaan Tallin | AI research better than better than Humans | - if we are talking about things that is able to do AI research better than better than humans uh then of course that would be um potentially manifested by the fact that we're dead | https://youtu.be/Dmh6ciu24v0?t=4614 | Yes | |
| Jaan Tallin | AI and LLMs are already being developed with help from AI | - the thing that I would be tracking is like how much uh in every successful every successive generation how much effort was kind of put in input by uh humans on how much was AI because it's already non-zero by AI when it comes to like designing the chips and architectures there's one more topic I want to hit on | https://youtu.be/Dmh6ciu24v0?t=4713 | Yes | |
| Jaan Tallin, Eliezer Yudkowsky | - researchers actually do not know how to reliably uh Point uh this black box to like any gold | ||||
| George Hotz | AI Could Turn the world into an RPG to help Humans? | - in an MMO RPG, the mission goal for how to win is listed at the top of the screen. AI could do that for Humans. Do we want that? | https://youtu.be/WS5wGal3ukw?si=Mx3mqIRVvRx0-cpv | ||
| elizer yudkowski | Moore's law for mad scientists - the IQ needed for people to become very dangerous drops my 2 IQ points each year... | https://youtu.be/R78mbtNeCvM?si=xlQvbBXXmEECBoEb | |||
| Conor Leahy | - it won't be dramatic. we'll just get more and more confused about what is real on the Internet. no one will know what's happening. and we all fall over dead one day and we won't know why | Yes | |||
| Dan Hendrycks | Natural Selection favors AIs | - Natural Selection favors AIs or Human and this could lead to Human extinction. | https://twitter.com/DanHendrycks/status/1651741353574019074 https://youtu.be/Fk7cMD44uWs?t=238 | Yes | |
| Eliezer Yudkowsky | Squiggle Maximizer (formerly "Paperclip maximizer") | - it would undergo an intelligence explosion: It would work to improve its own intelligence, where "intelligence" is understood in the sense of optimization power, the ability to maximize a reward/utility function—in this case, the number of paperclips. The AGI would improve its intelligence, not because it values more intelligence in its own right, but because more intelligence would help it achieve its goal of accumulating paperclips. Having increased its intelligence, it would produce more paperclips, and also use its enhanced abilities to further self-improve. Continuing this process, it would undergo an intelligence explosion and reach far-above-human levels. | https://www.lesswrong.com/tag/squiggle-maximizer-formerly-paperclip-maximizer | Yes | |
| Guillaume Verdon | The market will exhibit caution | - the market will exhibit caution. every organism company consumer is acting out of self-interest and they won't assign Capital to things that have negative utility to them | - "the problem is with the market is like you know there's not always perfect information there's manipulation there's a bad faith actors that mess with the system it's not it's not always a um rational and honest system". -- Lex Friedman | https://youtu.be/Fk7cMD44uWs?t=413 | No |
| Concrete Problems in AI Safety | - 1. Avoiding Negative Side Effects - 2. Avoiding Reward Hacking - 3. |
https://arxiv.org/abs/1606.06565 | Yes | ||
| Conor leahy | - they hack all the systems and confuse everyone | https://youtu.be/Cf_l6LNiXGI?si=LmM5JfGZuUnF4Fhq 22:50 | |||
| Jan Leike | Train up a Human Level AI and ask it how to align a SuperAGI | - Plan of how we want to approach the problem that involves training uh roughly human level alignment researcher that can work automatically and then ask that automated alignment researcher and to figure out how to align super intelligence -- Jan Leike https://youtu.be/Uk6-Rw5N_Dg?t=120 - A good way to like try to keep up with it is like to have your automated alignment residues that can actually like you know do thousands of years of equivalent work within like every week and that's just no way that that's gonna happen with humans -- Jan Leike https://youtu.be/Uk6-Rw5N_Dg?t=1559 |
- How good is it is the model as like spinning really clear and lies or like being deceptive of like pretending to do something or believe one thing and then actually wanting another I think another like really key capability here is self-exfiltration so how good would the model be like breaking the security precautions and like accessing its own weights and trying to copy it somewhere else on the internet um or persuading engineer with access to the weights to you know like download them and send them somewhere -- Jan Leike https://youtu.be/Uk6-Rw5N_Dg?t=517 - There's this desire to avoid fast takeoff right and I think there's this quote from planning for AGI and beyond that says it's possible that AGI capable enough to accelerate its own progress could cause major changes happen surprisingly quickly and then it says we think a slower takeoff is easier to make safe so one thing I wonder is like if we make this like really smart uh or you know human level alignment researcher that we then like effectively 10x or 100x or something the size of the alignment team does that end up playing into this like recursive self-improvement Loop? -- Daniel Filan AXRP https://youtu.be/Uk6-Rw5N_Dg?t=1476 |
https://youtu.be/Uk6-Rw5N_Dg?t=120 | No |