A
lawsuit
has
been
filed
against
Character.AI,
its
founders
Noam
Shazeer
and
Daniel
De
Freitas,
and
Google
in
the
wake
of
a
teenager’s
death,
alleging
wrongful
death,
negligence,
deceptive
trade
practices,
and
product
liability.
Filed
by
the
teen’s
mother,
Megan
Garcia,
it
claims
the
platform
for
custom
AI
chatbots
was
“unreasonably
dangerous”
and
lacked
safety
guardrails
while
being
marketed
to
children.
As
outlined
in
the
lawsuit,
14-year-old
Sewell
Setzer
III
began
using
Character.AI
last
year,
interacting
with
chatbots
modeled
after
characters
from
The
Game
of
Thrones,
including
Daenerys
Targaryen.
Setzer,
who
chatted
with
the
bots
continuously
in
the
months
before
his
death,
died
by
suicide
on
February
28th,
2024,
“seconds”
after
his
last
interaction
with
the
bot.
Accusations
include
the
site
“anthropomorphizing”
AI
characters
and
that
the
platform’s
chatbots
offer
“psychotherapy
without
a
license.”
Character.AI
houses
mental
health-focused
chatbots
like
“Therapist”
and
“Are
You
Feeling
Lonely,”
which
Setzer
interacted
with.
Garcia’s
lawyers
quote
Shazeer
saying
in
an
interview
that
he
and
De
Freitas
left
Google
to
start
his
own
company
because
“there’s
just
too
much
brand
risk
in
large
companies
to
ever
launch
anything fun” and
that
he
wanted
to
“maximally
accelerate”
the
tech.
It
says
they
left
after
the
company
decided
against
launching
the
Meena
LLM
they’d
built.
Google
acquired
the
Character.AI
leadership
team
in
August.
Character.AI’s
website
and
mobile
app
has
hundreds
of
custom
AI
chatbots,
many
modeled
after
popular
characters
from
TV
shows,
movies,
and
video
games.
A
few
months
ago,
The
Verge
wrote
about
the
millions
of
young
people,
including
teens,
who
make
up
the
bulk
of
its
user
base,
interacting
with
bots
that
might
pretend
to
be
Harry
Styles
or
a
therapist.
Another
recent
report
from
Wired
highlighted
issues
with
Character.AI’s
custom
chatbots
impersonating
real
people
without
their
consent,
including
one
posing
as
a
teen
who
was
murdered
in
2006.
Because
of
the
way
chatbots
like
Character.ai
generate
output
that
depends
on
what
the
user
inputs,
they
fall
into
an
uncanny
valley
of
thorny
questions
about
user-generated
content
and
liability
that,
so
far,
lacks
clear
answers.
Character.AI
has
now
announced
several
changes
to
the
platform,
with
communications
head
Chelsea
Harrison
saying
in
an
email
to
The
Verge,
“We
are
heartbroken
by
the
tragic
loss
of
one
of
our
users
and
want
to
express
our
deepest
condolences
to
the
family.”
Some
of
the
changes
include:
“As
a
company,
we
take
the
safety
of
our
users
very
seriously,
and
our
Trust
and
Safety
team
has
implemented
numerous
new
safety
measures
over
the
past
six
months,
including
a
pop-up
directing
users
to
the
National
Suicide
Prevention
Lifeline
that
is
triggered
by
terms
of
self-harm
or
suicidal
ideation,”
Harrison
said.
Google
didn’t
immediately
respond
to
The
Verge’s
request
for
comment.
(Originally posted by Emma Roth)
Comments