Cosmos Managed OS

"pure" Managed OS

Ben Kloosterman — Sat, 24 Jul 2010 09:17:05 GMT

Sooos will not be a pure managed OS , like Cosmos SharpOS or Mosa , or even Singularity ( which has a small amount of C++ code)...

I still feel that GC technology has some way to go and at present the performance penalty is too bad for certain functions. This is especially so for writes as most GCs todate implement a write barrier which can more that half write performance , that said as we go to higher core counts it may be more efficient to remove the write barrier.

In particularly 3D and some mathemetical modelling which need to work over a large mutable space.

However it may be possible in future and we can test a native lib wrapped in a service vs a pure service to see the impact on performance and only have critical services that suffer from GC costs running native . Anyway Sooos will take a practical approach it will have as much code as it practical type safe and memory safe and will move other performance critical native code when there is comparable managed code.

Modern UNIX /Linux compile , make files and dev environment

Ben Kloosterman — Tue, 08 Jun 2010 06:17:53 GMT

It is quite amazing what has been built in the UNIX space , look at Gentoo which will compile the entire system optomized for the machine . That being said i question is it worth it ? It seems to be that a significant time % of Unix development is spent on the dev environment.

The patches , the solving make file issues , the wrappers , the hacks , the conflicts , what works with what etc ? Compared to the Java and .NET world you just drop the code /assembly on the machine and it will compile and run regardless , at worst you have to add a new assembly. .

Build environments for most .NET projects tend to just be a list of files to compile and where to copy it. This is rather interesting , especially on windows where you far more complex installers with user dir , registries , dll registry etc though it has helped that .NET has dropped the use of registry for a simpler configuration environment .

The reason why its much simpler this is not the compilation environment etc but mainly because of

The C conditional compilation changes.
The increased use of runtime configuration options.
A reliance on a standard interface ( helped by a huge framework the interface of which doesnt change)
Denying the caller the chance to change or recompile the lib
The removal of the static compile concept.
A cross platform IR representation.

There is no reason Unix development cant adopt the same ideas eg

instead of delivering hard to compile code or single platform binaries you can distribute everything in say LVVM IR intermediate representation
Remove nearly all compile options and replace with runtime configuration options
Remove .configure
With the above 2 makefiles should be trivial .

It would make Unix development and deployment significantly easier and you could drop an assembly on any machine and it would compile optimaly for the machine. When enabling certain config option at run time you will be told if they are invalid

I doubt the above will happen but it would be a massive improvement ,the best option is probably if this is done through package distribution but the big difficult is pulling the code development out of the dark age or should that be the vi age .

OS memory protection - Is it worth it ?

Ben Kloosterman — Tue, 08 Jun 2010 06:13:12 GMT

It strikes me as interesting the horror people exhibit when you tell them you have disabled memory protection and hence the kernel transition. Lets look at this in more detail.

1) Most oS run with the majority of code with NO protection . In both Linux , OSX and Windows most drivers , services and the kernel exist in a single large domain representing the majority of the code base. Any error in the code can bring a machine down or cause a security issue.

2) Critical software like routers often run on Embedded systems most of which dont have or use an MMU and hence dont have protection.

3) Many servers run a single application. eg SQL server.

So what is bad about running with no protection.

1) Students & Developers on Time sharing systems . Obviously this is bad... quite a few students have brought machines down even with protection you dont need to make it worse.

2) Security . The whole address space is vissable which creates a security system.

So is it needed or not ?

Not on normal non shared systems , on personal computers or servers . However an extra layer of security is advisable and moving user apps to a sandbox such as Java or .NET will give protection without the cost..

Managed OS

Ben Kloosterman — Tue, 06 Apr 2010 03:10:21 GMT

Just an update on what I have been up to. I don’t have a lot of time and what little I have im spending on the kernel.

Some back ground

The key challenges to moderns OS are
1. Security ( more specifically hacking through 3rd party services likes Flash and limiting the damage from users allowing bad apps)
2. Reliability
3. Resiliency eg recovering from a critical failure

And to a lesser extent
4. Difficulty in managing change to an OS
5. Handling the Asynchronous nature of the internet
6. Handling multi cores efficiently

SOOOS ( Service Oriented Object Operating System) is a OS that scales from embedded to GP. It is not a simple research OS , the key goals are

*Scale from embedded to server ( this is easy with no MMU)
* Only allow managed user applications ( Java & .NET )
* A full capability security model with little ambient authority both improving code quality and the main security risks.
* TDD to increase reliability and ease of change
* Deeply asynchronous message passing IPC to improve security , maintenance , multi core scaling and performance
* Applications and Services are broken down into small services these services are load balanced and the OS can decide how many to run. These small services also allow greater reuse and flexibility as they are defined via an interface. Examples of these services are Crypto and XML parsing. The small services also increase parallelism.
* Any service can be made available to other machines.
* Self healing , services are monitored and terminated if they stop responding the system ensures there is always a minimum amount of services. .
* Performance at least 95% of Linux or windows , 105% is desirable.
* Kernelless. There is a TCB which I hope to keep under 100K lines.

doing I have continued documenting the design of my kernel ( 13 documents now) and discussing kernel designs with phil on #mosa and others.

I was using Cosmos but I think it will be a while before the compiler is at a decent state and that means heavy optimizations ( my guess 5 years Cosmos , 2 year mosa) After considering the above I was thinking for any OS to gain acceptance it needs to be fast and reliable which is difficult for us to achieve without a compiler in fact the first few non asm versions of UNIX failed ( based on B and C) until they had a good and fast compiler.

I then decided to write these components on Windows using a changeable HAL when mosa or Cosmos is ready I can change the Hal its ready to this end I wrote some IPC algorithms , locking mechanisms and a MM to see what they would look like . .I also don’t believe using corelib as a syslib is “safe” yet to write your own syslib for trusted components which I was doing is time consuming.

At this point I was working on shared data structures and for this would need an efficient ( non GC ) allocator so started looking at some Linux code , they have a good approach here with a Buddy page allocator for large amounts and SLAB/SLUB for lots of small objects ( just a large allocation say 2 Meg which hands out lots of small objects by marking them in use ) I then found a MMU_CONFIG option.

This is used by uCLinux designed for embedded systems , disable the MMU .
Features that are present
- A syslib ( uslib)
- Full Multi tasking
- Most of Linux works ,except calls to fork ( due to its use of copy on write paging) . This means things like the nvida driver
- Uses the standard Linux Kernel
- Support for both Position independent Code and remapping code on load
- Support for shared code
- Flat file format
- Flat memory model
- Support for many architectures including 680x0 ,arm etc

It than struck me If can get this and Mono working and then disable non .NET apps I have a managed OS which is fast , reliable and immediately useable ( eg package it up and use VS to create the managed apps and you already have an awesome embedded environment (though a bit big) ) .

We then add a new managed Asynch IPC , assembly install and compile , compile time range checks and capabilities and slowly move Linux drivers into managed user app drivers ( even a managed wrapper initially especially for the accelerated Nvidia and Radeon drivers)

Delete what is extra . Eventually there is less and less native code.

“Perfection is achieved not when you can’t add any more but when you cannot take any more away”

Result we start with a Linux derivative which is fast , stable and runs on many platforms and continually improve it to make it more secure , reliable and resilient.

Anyway that’s what im working on ( getting x86 Linux arch with no MMU and Mono running on it) .

Next Generation Operating System Why needed and Why now.

Ben Kloosterman — Sat, 13 Feb 2010 02:58:19 GMT

I am comming more and more to the conclusion that existing Operating System do not meet our needs nor can they be changed to handle these ( look at the cost for Windows 7 to improve parralelism cite) . This is not just Windows but also Symbian , Linux , OSX , Chrome etc. Im also saying UNIX is reaching its use by date.

Obviously the installed base is massive making change painfull but i believe it will come. The Niche markets of Security , , CLoud computing and Mobile devices especially dont feel the impact of an existing installed base as much.

While there are many smaller benefits the key benefits are

1. Performance on modern hardware . This is not just execution performance but also how applications are written for eg parallelism , while languages have made great strides in this area ( ie Hascal, F# and OCaml) they get stuck at the iPC level on the Operating System .

2. Asyncronous nature of modern communication most OS IPC is inefficient ( to what it could be ) for Asyncronous IPC. ( it requires extra threads , wastes a return etc).

3. Disk bound. All modern OS are heavily disk bound. While traditionally this makes sense we are rapidly aproaching diskless computers with internet ( or file server) storage . To some extent people have pushed for this but the OS nature has held as back. This especially affects UNIX with its everything is a file concept and existing file based apps. Can anyone argue against remoter boot of Proms and USB sticks and network/internet based access ?

Computers are regularly waiting on the DIsk , primarily for page swapping , yet page swapping is almost redundant today , the function can better be done with application swapping. Pageing results in a significant Operating system overhead in terms of performance but worse in terms of complexity , bugs and battery life / power usage.

4. Reliability . Significant increases have been achived in message based micro kernels the next generation operating systems will do this without the performance cost. Its worth noting that these message based Micro Kernel OS performance penalty has been getting worse due to the extra memory cost and memory speed has been growing slower than CPU.

5. Last but not least is security. There are 3 main issues

Operating systems need to accept the user will agree to things asked and need to limit the damage. This is possible with non intruesive methods as shown by capROS , Eros and Coyotos. These operating systems follow the POLP/POLA and applications have very low ambient authority. Hence any breach will have limited impact.
Existing operating system applications and services , can be made to crash and expose memory and data for futher exploitation , typesafe and memorysafe OS will massively reduce this most common form of attack.. The OS itself is only vulnerable through a specific bug and without things like buffer overuns these are much rarer.
UnIdentified applications . Apple has made strides in this area with approved apps though more for comercial reasons ( and since the approval proses is not independent people are starting to disable it ). Certificate signed .NET assemblies work pretty well as a measure of trust.

6. Resource management .

How often do computers crawl because a backup , Virus scan or index is running ? While these threads run at low priority there is no or little management of resources and since everything runs of the disk (see 3) things are waiting for all the time despite the fact that what they are waiting for is small.

Most of these things require applications to be written differently but note this is also being required for specific reasons eg parralelism , Minimizing GC pauses , Cloud applications , Asyncronous applications , new languages Caml/Hescal/F# and last but not least writing applications for POLP/POLA ( Principal of Least priviledge/Authority) all require changes in the way we write application. New generation operating systems should assist and encourage such programming.

In terms of why needed i think the recent Google attacks have shown that modern security is not effective enough.

The End of Linux ...and Windows

Ben Kloosterman — Tue, 04 Aug 2009 01:58:36 GMT

New generation of OS coming.

Operating Systems are about to enter their 4th generation with a revolutionary change. These being

Basic
TimeSharing
Hardware memory protection ( most were UNIX/MULTICS derived designs)
Managed OS. ( or OOOS)

While Singularity/Midori is a Manged OS it is still a traditional style OS. Not all 4th gen OS will be so traditional , things a operating system does like memory segmentation are not even needed.

A better name for these OS may be an Object Oriented Operating System or even Object Oriented Framework. Tradditional things like IPC (Asynch = events) and security can be done with language features (Interfaces ,Private members etc) especially using Capability OS techniques with the reference being a capability. In effect these OS are just huge OO applications with user apps being part of it.

The only thing regarding scheduling that may be required is preemption especially on single threaded CPU's. Yet this can still be done entirely in the OO framework .

Why do this ?
            Better performance than existing OS due to no IPC/memory overheads.
            More reliable , the OS can be easily tested and even proved.
            More secure. ( really more secure ,not like OSX/Linux due to fewer attacks)

All these things are especially needed in small internet linked devices ( including phones , Netbooks etc) , notice the latest iphone patch ? It will not be long before you will NEED a virus scanner and firewall on your phone as they will be attacked like windows.

I believe this is totally a revolutionary step as big as protected memory systems ( which are not needed now) were in building a stable OS remember early versions of Windows compared to NT /2000/XP .

However research is completely lacking . Consider the nameless OS im building with the Cosmos managed OS toolkit.

```
Managed OS - very little research
```

Single Address space - no modern research eg Multi core .

```
Asych IPC  - little modern research 
```

Capability Managed OS - no one has done it .

OO security for Capabilities - no one has done it.

Moving hardware access into user process and limit with capabilities - Very few people have done it most modern designs have user mode drivers but the hardware access is the HAL.

No Kernel calls - Minix is the only other OS I know of where IPC is used for Kernel calls .

No kernel - Even  "Nano" kernels have primatives and hardware in the Kernel .

How does this effect Linux ? I dont believe the open source community can move significant resources fast enough for a new completely incompatible Linux ( including all C and C++ user applications) and it needs a very dedicated individual to complete a kernel . Compare Linux ( mainly Linus initial effort ) to Gnu Herd ( an open source effort) . Note there are small open source managed OS efforts (Cosmos and MOSA) .

Why can Microsoft do it ? Because they have been working on this since 2002-2003 and have released a "Research Kernel" in 2005. They can pressure driver manufacturers and most .NET and Java applications can be fully supported and MS will move key apps like Office , IIS , Sql server to the new platform.

There is a real danger to open source where MS will sell a Midori OS to corporates like they did with NT and leave windows as a cheap alternative if Midori sells well. While this may not maximize short term revenue it will hold Microsofts dominant position and reinforce there sales for other applications eg Word for Windows 2 ( when windows was released) ,Office 2000 and SQL server for NT /win 2000.

Anyway Midori if the designers get it right will make its appearance in the next Major Windows Mobile version , it will be a great area as a lot of backward compatibility will be provided via Compact Framework.