Anthropic just launched a major upgrade to Claude Sonnet 3.5 and a new feature called "computer use" that allows AI to perform actions on a computer just lik...
That’s the crazy thing here, it is interacting with programs in a way that is wildly inefficient. At some point stuff like this will be properly integrated, and that both scares and excites me.
Yeah, using image recognition on a screenshot of the desktop and directing a mouse around the screen with coordinates is definitely an intermediate implementation. Open Interpreter, Shell-GPT, LLM-Shell, and DemandGen make a little more sense to me for anything that can currently be done from a CLI, but I’ve never actually tested em.
That’s the crazy thing here, it is interacting with programs in a way that is wildly inefficient. At some point stuff like this will be properly integrated, and that both scares and excites me.
Yeah, using image recognition on a screenshot of the desktop and directing a mouse around the screen with coordinates is definitely an intermediate implementation. Open Interpreter, Shell-GPT, LLM-Shell, and DemandGen make a little more sense to me for anything that can currently be done from a CLI, but I’ve never actually tested em.