Author offers question in his final words:
What do we want to want?
It's a cryptic question that only makes sense in the context of Homo sapiens' recently-found God status and approaching singularity and future step of evolution. With the ability to control even our own desires it converts the question of what do we want to become to what do we want to want.
I have often settled on and found comfort in the fact that the singularity represented the next major step of evolution in our species, and thus ensured the long-term of even permanent survival of Homo sapiens. Therefore I'm sanctifying the the survival of humanity itself as my "God" value. How far can we go?
But seldom have I considered what post-singularity life would even look like and whether it's a good thing. Does it even matter? From one perspective, it's simply progress and if "science" is my religion, then mere progress is the end goal.
From another perspective, if consciousness is the only thing of unique inherent value, as suggested by Nietzsche, then a singularity would undoubtedly mean an explosion of consciousness at a scale never before imagined. In that sense it's a win.
But returning to the question, if we are among the last of Homo sapiens, should we try to influence what kind of species comes next? Would it inherit the best of us or the worst of us? This is tricky because it feels a little circular to me. Who are we to say what is good and evil and based on what? It makes little sense to ask that question. It seems reasonable to assume that given a limitless ability to reason and experiment, that these AI gods would be liberated from the evolutionary pitfalls of human psychology in modern times and better than us at making value judgments.
For me the pursuit of knowledge itself is a reason to live and exist. I want to know what is currently beyond our understanding and comprehension. What is the nature of the universe? What are we inside of? What exists beyond what we can only perceive using current methods of detection and measurement? To me, humanity's ultimate goal should be the full reconciliation of modern scientific theories into a single unified theory of everything. Will there ever be stopping point?
Maybe the ultimate goal is for consciousness or "life" to fully comprehend and control the fate of the universe itself. The idea however that there could ever be a limit seems a paradox. How can there be a "wall" where the known universe "ends"? How can there ever be something that is nothing?
If some great consciousness does eventually reign in the forces of the universe and nature, that essentially makes the consciousness an all-knowing, superhuman God, which ironically is an end product of science. Maybe there is one already? Chicken or the egg. But beyond this, what is next? There must be a domain or an order beyond the known universe that is infinite, yet at this point it gets blurry to try to guess at what anything like that is or could be.
So in the end what is the point? What is the meaning of life? There is none. Just be. But how do you "be"? Feel, learn, love.