Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: ClamAV: devel

Bytecode interpreter

 

 

ClamAV devel RSS feed   Index | Next | Previous | View Threaded


dfs at roaringpenguin

Mar 10, 2010, 12:54 PM

Post #1 of 12 (5548 views)
Permalink
Bytecode interpreter

Hi,

I noticed the announcement of the bytecode interpreter in the 0.96-rc1
announcement.

That feature took me utterly by surprise.

Could anyone provide a use-case for it? I'm at a loss as to why a
security tool should allow signature writers to be able to inject
arbitary executable code. (Yes, I know the bytecode has all kinds of
security checks and is limited in what it can do, but so does/did Java
and there were still many bugs found in the Java sandbox.)

And a security tool that requires (or at least can use) a C compiler
at run-time boggles the mind. I guess we either have to install
a C compiler or live with the slower bytecode interpreter.

So...

Why do we need the bytecode interpreter? Can we disable it if we decide
the cons outweigh the pros?

Regards,

David.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


edwintorok at gmail

Mar 11, 2010, 4:17 AM

Post #2 of 12 (5398 views)
Permalink
Re: Bytecode interpreter [In reply to]

On 2010-03-10 22:54, David F. Skoll wrote:
> Hi,
>
> I noticed the announcement of the bytecode interpreter in the 0.96-rc1
> announcement.
>
> That feature took me utterly by surprise.
>
> Could anyone provide a use-case for it?

Hi,

Right now the only detections one can write are pattern-based.
You can't write heuristic detections, you can't write unpackers, you
can't support new file formats, and you can't do more complex analysis
than pattern matching.
The bytecode tries to offer the possibility to do the above, without
releasing a new engine update each time.

> I'm at a loss as to why a
> security tool should allow signature writers to be able to inject
> arbitary executable code. (Yes, I know the bytecode has all kinds of
> security checks and is limited in what it can do, but so does/did Java
> and there were still many bugs found in the Java sandbox.)
>
> And a security tool that requires (or at least can use) a C compiler
> at run-time boggles the mind.

It doesn't use a C compiler at runtime.
It has both an interpreter, and a JIT that can run the bytecode.
The JIT creates machine code from bytecode, not from C code.

> I guess we either have to install
> a C compiler or live with the slower bytecode interpreter.

No, the JIT is already include in libclamav, just run clamconf and see
if it reports the JIT feature.
It won't use an external compiler.

>
> So...
>
> Why do we need the bytecode interpreter? Can we disable it if we decide
> the cons outweigh the pros?

freshclam.conf, "Bytecode no". That will prevent bytecode.cvd from being
downloaded.

Best regards,
--Edwin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


clamav-devel at jubileegroup

Mar 11, 2010, 5:29 AM

Post #3 of 12 (5395 views)
Permalink
Re: Bytecode interpreter [In reply to]

Hi there,

On Thu, 11 Mar 2010 David F. Skoll wrote:

> I noticed the announcement of the bytecode interpreter in the 0.96-rc1
> announcement.
> ...
> Why do we need the bytecode interpreter? Can we disable it if we decide
> the cons outweigh the pros?

I was about to write something along these lines when Mr. Skoll's post
arrived. The very idea of a bytecode interpreter in ClamAV gives me
the creeps. It sounds like a whole bunch of vulnerabilities waiting
to happen. I'd like to add my voice to those who want an easy way to
disable it - I can see nothing in the clamd.conf man page for 0.96-rc1
which offers any solace.

In the same man page there are a couple of small formatting errors in
the bold attributes for LocalSocketGroup and LocalSocketMode.

--

73,
Ged.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


rbgarga at gmail

Mar 11, 2010, 5:44 AM

Post #4 of 12 (5394 views)
Permalink
Re: Bytecode interpreter [In reply to]

On Thu, Mar 11, 2010 at 10:29 AM, G.W. Haywood
<clamav-devel [at] jubileegroup> wrote:
> Hi there,
>
> On Thu, 11 Mar 2010 David F. Skoll wrote:
>
>> I noticed the announcement of the bytecode interpreter in the 0.96-rc1
>> announcement.
>> ...
>> Why do we need the bytecode interpreter?  Can we disable it if we decide
>> the cons outweigh the pros?
>
> I was about to write something along these lines when Mr. Skoll's post
> arrived.  The very idea of a bytecode interpreter in ClamAV gives me
> the creeps.  It sounds like a whole bunch of vulnerabilities waiting
> to happen.  I'd like to add my voice to those who want an easy way to
> disable it - I can see nothing in the clamd.conf man page for 0.96-rc1
> which offers any solace.
>
> In the same man page there are a couple of small formatting errors in
> the bold attributes for LocalSocketGroup and LocalSocketMode.

IIRC, you can use --enable-llvm=no at ./configure to disable.

--
Renato Botelho
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


tkojm at clamav

Mar 11, 2010, 6:26 AM

Post #5 of 12 (5398 views)
Permalink
Re: Bytecode interpreter [In reply to]

On Thu, 11 Mar 2010 13:29:16 +0000 (GMT)
"G.W. Haywood" <clamav-devel [at] jubileegroup> wrote:

> Hi there,
>
> On Thu, 11 Mar 2010 David F. Skoll wrote:
>
> > I noticed the announcement of the bytecode interpreter in the 0.96-rc1
> > announcement.
> > ...
> > Why do we need the bytecode interpreter? Can we disable it if we decide
> > the cons outweigh the pros?
>
> I was about to write something along these lines when Mr. Skoll's post
> arrived. The very idea of a bytecode interpreter in ClamAV gives me
> the creeps. It sounds like a whole bunch of vulnerabilities waiting
> to happen.

Due to security reasons all bytecodes need to be digitally signed,
so no 3rd parties will be able to inject any code into your installations.
When it comes to vulnerabilities, they will not be that critical as
vulnerabilities in the regular code since all bytecodes can be remotely
fixed/removed.

> I'd like to add my voice to those who want an easy way to
> disable it - I can see nothing in the clamd.conf man page for 0.96-rc1
> which offers any solace.

As Edwin already described, you just set the "Bytecode" option to "no"
in freshclam.conf.

> In the same man page there are a couple of small formatting errors in
> the bold attributes for LocalSocketGroup and LocalSocketMode.

Thanks, this will be fixed in the next release

Regards,

--
oo ..... Tomasz Kojm <tkojm [at] clamav>
(\/)\......... http://www.ClamAV.net/gpg/tkojm.gpg
\..........._ 0DCA5A08407D5288279DB43454822DC8985A444B
//\ /\ Thu Mar 11 15:12:49 CET 2010
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


edwintorok at gmail

Mar 11, 2010, 6:35 AM

Post #6 of 12 (5398 views)
Permalink
Re: Bytecode interpreter [In reply to]

On 2010-03-11 15:44, Renato Botelho wrote:
>
> IIRC, you can use --enable-llvm=no at ./configure to disable.
>

That just disables the JIT, not the interpreter.


On 2010-03-11 16:26, Tomasz Kojm wrote:
> On Thu, 11 Mar 2010 13:29:16 +0000 (GMT)
> "G.W. Haywood" <clamav-devel [at] jubileegroup> wrote:
>
>> Hi there,
>>
>> On Thu, 11 Mar 2010 David F. Skoll wrote:
>>
>>> I noticed the announcement of the bytecode interpreter in the 0.96-rc1
>>> announcement.
>>> ...
>>> Why do we need the bytecode interpreter? Can we disable it if we decide
>>> the cons outweigh the pros?
>> I was about to write something along these lines when Mr. Skoll's post
>> arrived. The very idea of a bytecode interpreter in ClamAV gives me
>> the creeps. It sounds like a whole bunch of vulnerabilities waiting
>> to happen.
>
> Due to security reasons all bytecodes need to be digitally signed,
> so no 3rd parties will be able to inject any code into your installations.
> When it comes to vulnerabilities, they will not be that critical as
> vulnerabilities in the regular code since all bytecodes can be remotely
> fixed/removed.

Yes, and let me explain some of the other security features:
- bytecode can only call functions it defines itself, and a limited
ClamAV API (see libclamav/bytecode_api.h), no syscalls
- no direct access to the filesystem, it can only read the currently
scanned file (via the API), and write to a temporary file via the API
(when unpacking)
- no arbitrary memory access, bounds of all accesses must be known,
bounds checks inserted by the compiler, or libclamav itself (see
BytecodeSecurity in clamd.conf)
- although the above should be enough, there is also stack smashing
protection in the JITed code (which simply aborts the bytecode, not clamd)

>
>> I'd like to add my voice to those who want an easy way to
>> disable it - I can see nothing in the clamd.conf man page for 0.96-rc1
>> which offers any solace.
>
> As Edwin already described, you just set the "Bytecode" option to "no"
> in freshclam.conf.
>
>> In the same man page there are a couple of small formatting errors in
>> the bold attributes for LocalSocketGroup and LocalSocketMode.
>
> Thanks, this will be fixed in the next release
>
> Regards,
>

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


dfs at roaringpenguin

Mar 11, 2010, 6:47 AM

Post #7 of 12 (5400 views)
Permalink
Re: (no subject) [In reply to]

Török Edwin wrote:

> Right now the only detections one can write are pattern-based. You
> can't write heuristic detections, you can't write unpackers, you
> can't support new file formats, and you can't do more complex
> analysis than pattern matching. The bytecode tries to offer the
> possibility to do the above, without releasing a new engine update
> each time.

Well, there's a fundamental philosophical problem here.

You've essentially introduced a software update mechanism that bypasses
the normal way to install ClamAV.

Furthermore, you're trying to write complex algorithms in byte code rather
than C. (Or do you have a high-level language that compiles down to
byte code?) This will require a completely new set of coding and
debugging skills.

Also, the *only* thing protecting us from malicious byte-code is your GPG
key. I hope you keep it safe. And nothing will protect us from buggy
byte code. Looking at the bytecode implementation, I found an easy
way to DoS ClamAV... do we really want that ability?

>> And a security tool that requires (or at least can use) a C compiler
>> at run-time boggles the mind.

> It doesn't use a C compiler at runtime.

Really? Why do the release notes say:

The following packages are optional, but required for bytecode JIT support:
GCC C and C++ compilers (minimum 4.1.3, recommended 4.3.4 or newer)

Do you mean that GCC is required only at build time?

Anyway... I don't like large and complex (barely-commented, full of
mysterious hard-coded constants) code being added to a security tool,
especially when that large and complex code implements a
(Turing-complete?) computer. I just worry that some future version of
ClamAV will require the bytecode interpreter, similar to how 0.94 was
EOL'd because of signature engine limitations. I also worry that
there will be pressure to expose more and more API functions to the
bytecode interpreter---it's so tempting when you just need "one more
thing" to implement a new detection algorithm.

Regards,

David.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


edwintorok at gmail

Mar 11, 2010, 6:54 AM

Post #8 of 12 (5394 views)
Permalink
Re: (no subject) [In reply to]

On 2010-03-11 16:47, David F. Skoll wrote:
> Török Edwin wrote:
>
>> Right now the only detections one can write are pattern-based. You
>> can't write heuristic detections, you can't write unpackers, you
>> can't support new file formats, and you can't do more complex
>> analysis than pattern matching. The bytecode tries to offer the
>> possibility to do the above, without releasing a new engine update
>> each time.
>
> Well, there's a fundamental philosophical problem here.
>
> You've essentially introduced a software update mechanism that bypasses
> the normal way to install ClamAV.
>
> Furthermore, you're trying to write complex algorithms in byte code rather
> than C. (Or do you have a high-level language that compiles down to
> byte code?) This will require a completely new set of coding and
> debugging skills.
>
> Also, the *only* thing protecting us from malicious byte-code is your GPG
> key. I hope you keep it safe. And nothing will protect us from buggy
> byte code. Looking at the bytecode implementation, I found an easy
> way to DoS ClamAV... do we really want that ability?

If you found a bug, please open a bugreport.

>
>>> And a security tool that requires (or at least can use) a C compiler
>>> at run-time boggles the mind.
>
>> It doesn't use a C compiler at runtime.
>
> Really? Why do the release notes say:
>
> The following packages are optional, but required for bytecode JIT support:
> GCC C and C++ compilers (minimum 4.1.3, recommended 4.3.4 or newer)
>
> Do you mean that GCC is required only at build time?

Build time, yes. The documentation should probably make that more explicit.

>
> Anyway... I don't like large and complex (barely-commented, full of
> mysterious hard-coded constants) code being added to a security tool,
> especially when that large and complex code implements a
> (Turing-complete?) computer. I just worry that some future version of
> ClamAV will require the bytecode interpreter, similar to how 0.94 was
> EOL'd because of signature engine limitations. I also worry that
> there will be pressure to expose more and more API functions to the
> bytecode interpreter---it's so tempting when you just need "one more
> thing" to implement a new detection algorithm.
>
> Regards,
>
> David.
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


dfs at roaringpenguin

Mar 11, 2010, 11:26 AM

Post #9 of 12 (5395 views)
Permalink
Re: Bytecode interpreter [In reply to]

Tomasz Kojm wrote:

> Due to security reasons all bytecodes need to be digitally signed,
> so no 3rd parties will be able to inject any code into your installations.

I believe this is the same security model used by Microsoft for Active X.
(NOTE: I am in no way implying that your bytecode interpreter is as
dangerous. I am implying that anyone can make an honest mistake and
sign buggy code, or have his private key compromised.)

> When it comes to vulnerabilities, they will not be that critical as
> vulnerabilities in the regular code since all bytecodes can be remotely
> fixed/removed.

OK... here's another question: ClamAV is licensed under the GPL. Your
bytecode programs are distributed in object-code format.

Will you make the corresponding source code available? What language
is the source code written in? It makes me nervous to see a GPLd
project starting to rely heavily on code for which source may or may
not be available. Unless you are careful, this could be the beginning
of ClamAV's changeover to a non-open-source system.

When a new release of Clam comes out, I download it, import it into my
git tree, and carefully go through the "git diff" output. I can see
what's new. When new bytecodes are released... *shrug* I have no idea
what they do.

Regards,

David.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


tkojm at clamav

Mar 11, 2010, 12:24 PM

Post #10 of 12 (5395 views)
Permalink
Re: Bytecode interpreter [In reply to]

On Thu, 11 Mar 2010 14:26:07 -0500
"David F. Skoll" <dfs [at] roaringpenguin> wrote:

> Tomasz Kojm wrote:
>
> > Due to security reasons all bytecodes need to be digitally signed,
> > so no 3rd parties will be able to inject any code into your installations.
>
> I believe this is the same security model used by Microsoft for Active X.
> (NOTE: I am in no way implying that your bytecode interpreter is as
> dangerous. I am implying that anyone can make an honest mistake and
> sign buggy code, or have his private key compromised.)
>
> > When it comes to vulnerabilities, they will not be that critical as
> > vulnerabilities in the regular code since all bytecodes can be remotely
> > fixed/removed.
>
> OK... here's another question: ClamAV is licensed under the GPL. Your
> bytecode programs are distributed in object-code format.
>
> Will you make the corresponding source code available?

yes, the bytecodes will embed the source code and the new tool
called "clambc" shipped with 0.96 can display the corresponding
source code with --printsrc

> What language is the source code written in?

In a C-like language

--
oo ..... Tomasz Kojm <tkojm [at] clamav>
(\/)\......... http://www.ClamAV.net/gpg/tkojm.gpg
\..........._ 0DCA5A08407D5288279DB43454822DC8985A444B
//\ /\ Thu Mar 11 21:21:59 CET 2010
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


clamav-devel at jubileegroup

Mar 12, 2010, 8:54 AM

Post #11 of 12 (5384 views)
Permalink
Re: Bytecode interpreter [In reply to]

Hi there,

On Fri, 12 Mar 2010 Tomasz Kojm wrote:

> > G.W. Haywood wrote:
> > I'd like to add my voice to those who want an easy way to disable
> > [the bytecode interpreter] - I can see nothing in the clamd.conf
> > man page for 0.96-rc1 which offers any solace.
>
> As Edwin already described, you just set the "Bytecode" option to "no"
> in freshclam.conf.

I'm starting to wonder if you guys shouldn't get out more.

Simply giving the bytecode interpreter nothing to interpret is not
acceptable. I don't want to just be able to give the interpreter
nothing to do; I would want to be able to disable it, so that it can't
do anything, even (especially!) if it is given something to do.

You'll understand why I didn't look in the freshclam.conf man page; I
was thinking more along the lines of an option to the daemon at the
time it is started, or perhaps - much better - a 'configure' option, so
that the interpreter code isn't even built into the, er, daemon binary.

--

73,
Ged.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net


edwintorok at gmail

Mar 12, 2010, 9:09 AM

Post #12 of 12 (5412 views)
Permalink
Re: Bytecode interpreter [In reply to]

On 03/12/2010 06:54 PM, G.W. Haywood wrote:
> Hi there,
>
> On Fri, 12 Mar 2010 Tomasz Kojm wrote:
>
>>> G.W. Haywood wrote:
>>> I'd like to add my voice to those who want an easy way to disable
>>> [the bytecode interpreter] - I can see nothing in the clamd.conf
>>> man page for 0.96-rc1 which offers any solace.
>> As Edwin already described, you just set the "Bytecode" option to "no"
>> in freshclam.conf.
>
> I'm starting to wonder if you guys shouldn't get out more.
>
> Simply giving the bytecode interpreter nothing to interpret is not
> acceptable. I don't want to just be able to give the interpreter
> nothing to do; I would want to be able to disable it, so that it can't
> do anything, even (especially!) if it is given something to do.

How would you give it something to do if you didn't load any bytecodes?

>
> You'll understand why I didn't look in the freshclam.conf man page; I
> was thinking more along the lines of an option to the daemon at the
> time it is started, or perhaps - much better - a 'configure' option, so
> that the interpreter code isn't even built into the, er, daemon binary.

I think a configure option is possible, it would work the same way as
--enable-llvm/--disable-llvm builds/links either libclamav/c++ or
libclamav/bytecode_nojit.c: there could be a libclamav/bytecode_disabled.c

Best regards,
--Edwin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

ClamAV devel RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.