There is no definitive answer as yet and you will obtain a different understanding by different parties depending on where you live and who you are talking to.
There is a small fine or penalty for not complying with the compulsory schooling requirement. In the past, there have been a few cases of parents who have had to pay that fine in court, so some infer from the mere existence of the fine that homeschooling is illegal. But we should question that assumption. (The fine is rarely applied and amounts to less than ten thousand yen.)
One prefecture's kyoiku inkai (educational authority) has actually publicly sanctioned homeschooling (according to press reports) and will send teachers to the home, you could also infer that that is a very good precedent for considering homeschooling to be legal. Provided you establish eligibility and are accountable to some local authority for your homeschooling activity.
There are thousands of futoko (school refusal) kids who do not attend public schools (and many of them, no school of any sort at all), but none of their parents have been hauled off to court. We know from the White Papers that the authorities have accepted the phenomenon as a social problem and are trying their best to work with parents to bring kids back into the schoolpens. They do this by establishing counseling rooms, encourage the kids receive schooling in alternative ways, such as Tokyo Shure (free schools), correspondence courses, including what we know as home education. However, while futoko kids who are schooled at home are a generally acceptable situation, the response to homeschooled kids (homeschooled for reasons other than of the kid' refusal to attend school) will vary from case to case, from prefecture to prefecture. Sometimes, a "no, that's not allowed" is a response simply out of ignorance on the part of the kyoiku inkai official and there is no reason to accept that response at face value.
The real question is the interpretation of the compulsory education requirement and how well you, as a parent, are meeting the corresponding duty (under the Japanese constitution) to ensure that right is not violated. Is it the right to education, a right of the child to receive access to education or is it legalistically interpreted as the corresponding duty of the parent to send the child to a formal school institution. If you look at the history of education in Japan since postwar educational laws were drafted (the laws have not changed since the Americans helped redraft the Constitution and educational law), we know that access to jr high and high school education used to be a privilege rather than a right. The numbers of students attending jr-high schools during the postwar years (understandably so -- poverty and all) used to be really low. There was a well-known phenomenon of crusading parents lobbying to have more jr - high schools built, so that all children could attend the schools should they want to -- ie. that the right to access to free education could be realized.
It is thus reasonable to say that a homeschooling parent is not in violation of the duty to ensure that the child receives an education if he doesn't send the child to a formal jr or high school. (The UN has clarified in recent years, in situations where the exercise of the right of the child conflicts with the rights of parents -- the solution is what is in the best interest of the child. In many courts of law, this is the best principle to argue, and homeschoolers should remember that). However, (as mentioned before) since this generation of majority of Japanese people (lay people or educational officials included) know of no exceptions to formal schooling for middle and high schoolers, and homeschooling is still a relatively unknown phenomenon, many Japanese naturally assume formal schooling is required by law and that homeschooling is illegal.
n Japan, the commonest ways in which homeschooling (we know from anecdotes or stories shared by people) has gained acceptance or has been allowed by the authorities occur in the following forms:
-- Most people have succeeded in persuading local educational authorities that homeschooling is not a violation of the compulsory schooling requirement, but that rather it is really a legal exception -- private schools are an exception to public schools. In some places, authorities have usually allowed homeschooling where the parent says the kids are enroled in a private school, in most cases they have to show some evidence -- sometimes registering with the ward or showing visiting kyoiku inkai and/or school officials, the materials or curriculum used is enough -- sometimes more is required like registration with an umbrella school (some use common homeschooling options from the US, some use a local Japanese institution that "covers homeschoolers" with a diploma from the US, others in Osaka have used UK curricula with legal covering from UK institutions.
-- Another successful way not to fall foul of the authorities is to convince your local public school head that you are homeschooling your kids because of your exceptional circumstances (either you are a bicultural family -- there is an educational by-law that provides exemption for such families; you are planning to send your kids overseas to foreign schools in the near future, etc) this has been often been accepted and has been sufficient for homeschooled children to remain enroled in the local public school, and eventually graduate with a local public school diploma. It is a popular way for homeschoolers to go, because local kyoiku inkai officials generally defer to the local public school officials on such matters, and there will be less bureaucratic interference or home visits; other advantages range from free textbooks, possibilities of even choosing activities in which you can participate in the local school such as kokugo classes, sports, clubs, etc (hence kids are in some measure integrated into the local community). The Tazonos (among others) are one such family who succcessfully negotiated this combination of homeschooling and time in the public school for their son.
The Monbusho's recent policies have clearly been in favor of alternative forms of education, other than public schools, such as long-distance correspondence courses; They have also made it clear from White Papers that they want more alternative forms of education for futoko kids as well as that they want MORE parental involvement in children's education (though stopping short of mentioning homeschooling). However, from the history of education once again, we know that national monbusho policies either take time to filter down to implementation by schools, also that there are many schools who regularly ignore any initiatives by the government, and doing things the way they have always done them. Some politicians have failed miserably in the past to implement certain policies on account of because of the recalcitrant attitude of school administrators and teachers. Policy is a political game and control is not always in the hands of policy-makers. So I have mentioned in previous posts, one's success in presenting one's right to homeschool must take into account where the power lies, basically, for homeschoolers -- power lies with those who have the power to deny you that right to homeschool -- usually it will be your local public school head or local ward kyoiku inkai officials.
So is homeschooling legal? My answer is that while it is an evolving area with no clear precedents in court, and since laws often crystallise from established practice, custom or precedentwe should say it is legal (until clearly declared illegal by either the Monbusho or the courts) because homeschooling activities are being practised, allowed and accepted by authorities in many places. Local kyoiku inkais will issue a postcard informing all residents in the ward of the date when they are to enrol their kids in the nearest public school. If you wish to homeschool, rather than ignoring the postcard, you must be prepared to "negotiate" with the local public school principal and/or the kyoiku inkai officials how you plan to ensure that your child receives a proper education. You must take care to ensure that you have complied with any local requirements and are accountable to the various local authorities. Depending on how "difficult" the authorities in your ward are going to be, this often means you should be able to show clear plans to homeschool, a curriculum and evidence of materials to be used.
Background historical note on why Japanese psyche is one that places top priority on formal and institutional schooling:
Japan has the highest rates of school attendance through to college in the world today (rivalling even Germany which is the only clear-cut example we know in the world of a country in which compulsory schooling means homeschooling is illegal). If you look further back through the centuries, unlike in western civilizations, Japan is unusual in that historically, it had the largest most highly educated population from ancient times. Beyond the Heian age when education still lay exclusively in the ranks of the nobility and imperial family, education spread across the classes due to the proliferation of temple schools, development of trade and craftsman guilds. Unlike China and other countries of the time where education was reserved for the nobility or upper classes, basic education was widespread among the masses (peasants and slaves excepted). From the merchant classes, to tradesmen and craftsmen apprentices and families, to samurai families to those who serviced the huge religious temple complexes -- education in terms of learning of writing, copying of Chinese classics, or Buddhist scripts and mastery of staple arithmetic basics of the time was the norm, except for a relatively short period, when the samurai discipline degenerated during the years when samurais and their families, daimyo lords were promoted by virtue of whims of the shogun instead by ability. By the Edo era, when Edo was the largest metropolis in the world at the time, it is certainly well known that the Japanese populace was the most educated civilization in the world -- the legendary samurai classics, haiku of Basho and Issa, the blockprinting art, kabuki, etc. Of course, peasants and slaves did not have such privileges, but upon occasion, we hear of some personnage of humble birth who gets adopted into the samurai family or becomes a monk, those were the main routes to learning of the times for the humble and non-entities of society.
Disclaimer: In the event you are planning to do battle with authorities in court over your rights to homeschool, please do rely on independent professional counsel.